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Mission 

Our  mission  is  to  provide  independent,  relevant,  and  timely 
oversight  of  the  Department  of  Defense  that:  supports  the 
warfighter;  promotes  accountability,  integrity,  and  efficiency; 
advises  the  Secretary  of  Defense  and  Congress;  and  informs 

the  public. 

Vision 

Our  vision  is  to  be  a  model  oversight  organization  in  the  federal 
government  by  leading  change,  speaking  truth,  and  promoting 
excellence;  a  diverse  organization,  working  together  as  one 
professional  team,  recognized  as  leaders  in  our  field. 


Fraud,  Waste  and  Abuse 

HOTLINE 

1.800.424.9098  •  www.dodig.mil/hotline 


For  more  information  about  whistleblower  protection,  please  see  the  inside  back  cover. 


Results  in  Brief 

Quality  Assurance  Assessment  of  the 
F-35  Lightning  II  Program 


September  30,  2013 


Objective 

We  performed  an  evaluation  of  the  F-35  Light¬ 
ning  II  Program  (F-35  Program]  by  conducting 
a  series  of  quality  assurance  assessments  of 
the  Joint  Program  Office,  prime  contractor,  and 
major  subcontractors.  We  assessed  conformity 
to  the  contractually  required  Aerospace 
Standard  (ASJ9100,  "Quality  Management 
Systems  -  Requirements  for  Aviation,  Space 
and  Defense  Organizations,”  contractual 
quality  assurance  clauses,  and  internal  quality 
assurance  processes  and  procedures  for  the 
following  six  contractors: 

•  Lockheed  Martin  Aeronautics  Company, 
Fort  Worth,  Texas  (Prime  Contractor  and 
Aircraft  Integrator]; 

•  Northrop  Grumman  Aerospace  Systems, 
El  Segundo  and  Palmdale,  California 
(Center  Fuselage  Integrator]; 


Findings 

The  F-35  Program  did  not  sufficiently  implement  or  flow  down 
technical  and  quality  management  system  requirements  to  prevent 
the  fielding  of  nonconforming  hardware  and  software.  This  could 
adversely  affect  aircraft  performance,  reliability,  maintainability,  and 
ultimately  program  cost.  Lockheed  Martin  Aeronautics  Company 
(Lockheed  Martin]  and  its  subcontractors  did  not  follow  disciplined 
AS9100  Quality  Management  System  practices,  as  evidenced  by 
363  findings,  which  contained  719  issues. 

The  Joint  Program  Office  did  not: 

•  Ensure  that  Lockheed  Martin  and  its  subcontractors 
were  applying  rigor  to  design,  manufacturing,  and  quality 
assurance  processes. 

•  Flow  down  critical  safety  item  requirements. 

•  Ensure  that  Lockheed  Martin  flowed  down  quality  assurance 
and  technical  requirements  to  subcontractors. 

•  Establish  an  effective  quality  assurance  organization. 


•  BAE  Systems,  Samlesbury,  United  King¬ 
dom  (Aft  Fuselage  Integrator]; 


•  Ensure  that  the  Defense  Contract  Management  Agency 
perform  adequate  quality  assurance  oversight. 


•  L-3  Display  Systems,  Alpharetta,  Georgia 
(Panoramic  Cockpit  Display  System]; 

•  Honeywell  Aerospace,  Yeovil,  United 
Kingdom  (On-Board  Oxygen  Generation 
System];  and 


In  addition,  the  Defense  Contract  Management  Agency  did  not: 

•  Sufficiently  perform  Government  quality  assurance  oversight 
of  F-35  contractors. 

Recommendations 


•  United  Technologies  Corporation,  Aero¬ 
space  Systems,  Fort  Worth,  Texas, 
and  Independence,  Ohio  (Landing 
Gear  System], 


The  Joint  Program  Office  should: 

•  Ensure  compliance  with  AS9100  throughout  the  F-35 
supply  chain. 


Visit  us  on  the  web  at  www.dodig.mil 
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Recommendations  Continued 

•  Ensure  that  Lockheed  Martin  approves  all  design  and 
material  review  board  changes  and  variances  with 
Government  concurrence. 

•  Perform  process  proofing  of  all  critical  processes  to 
include  first  article  inspections. 

•  Modify  its  contracts  to  include  a  quality  escape* 
clause  to  ensure  the  Government  does  not  pay  for 
nonconforming  product. 

•  Assess  the  impacts  and  risks  to  all  delivered  aircraft  for 
all  findings. 

•  Implement  an  aviation  critical  safety  item  program  that 
meets  the  requirements  of  Public  Law  and  DoD  policy, 
which  would  include  flow  down  of  requirements  for  a 
critical  safety  item  program  to  Lockheed  Martin  and 
its  subcontractors. 

•  Assess  the  impacts  and  risks  to  all  delivered  aircraft  for 
critical  safety  item  deficiencies. 

•  Perform  technical  and  quality  assurance  require¬ 
ment  flow  down  and  verification  throughout  the  F-35 
supply  chain. 

•  Establish  an  independent  quality  assurance  organiza¬ 
tion,  which  has  the  authority  and  resources  to  enforce 
the  AS9100  standard  and  F-35  product  quality. 

•  Revise  the  Defense  Contract  Management  Agency 
memorandum  of  agreement  to  include  explicit  quality 
assurance  oversight  requirements. 


A  quality  escape  is  nonconforming  material  that  has  entered  the  product, 
supply  chain,  or  proceeded  beyond  the  acceptance  process. 


•  Ensure  that  Defense  Contract  Management  Agency  is 
performing  quality  assurance  oversight  commensurate 
with  product  criticality. 

The  Defense  Contract  Management  Agency  should: 

•  Provide  a  comprehensive  quality  assurance  oversight 
plan  for  Joint  Program  Office  approval  to  be  included  in 
the  memorandum  of  agreement. 

•  Audit  the  execution  of  the  quality  assurance  oversight 
plan  throughout  the  F-35  supply  chain. 

Management  Comments  and 
Our  Response 

On  August  23,  2013,  the  Joint  Program  Office  and  the 
Defense  Contract  Management  Agency  responded  to  the 
findings  and  recommendations  in  the  report.  The  Joint 
Program  Office  agreed  with  eight  recommendations, 
partially  agreed  with  two,  and  disagreed  with  one.  The  Joint 
Program  Office  stated  that  it  does  not  have  the  resources 
to  perform  process  proofing  of  all  critical  processes  nor 
has  the  responsibility  or  resources  to  perform  requirement 
flow  down  verification  throughout  the  F-35  supply  chain. 
However,  we  disagree  because  it  is  the  J oint  Program  Office’s 
responsibility  to  ensure  contractual  compliance  to  prevent 
nonconformances.  It  is  also  the  responsibility  of  the  Joint 
Program  Office  to  update  the  contract  if  the  requirements 
are  deficient. 

It  was  also  our  recommendation  that  Joint  Program  Office 
establish  an  independent  quality  assurance  organization 


ii  I  DODIG-2013-140  (Project  No.  D2012-DT0TAD-0003.000) 


Results  in  Brief 

Quality  Assurance  Assessment  of  the 
F-35  Lightning  II  Program 


Management  Comments  and  Our  Response  Continued 


reporting  to  the  Program  Manager.  The  Joint  Program  Office 
disagreed  stating  that  the  Defense  Contract  Management 
Agency  performs  the  role  of  the  independent  quality 
assurance  organization  for  the  F-35.  We  disagree  because 
the  Defense  Contract  Management  Agency  is  not 
accountable  for  program  quality  assurance  goals.  An 
independent  quality  assurance  organization  reporting 
directly  to  the  Program  Manager  would  ensure  that 
performance  and  reliability  objectives  are  met. 

The  Defense  Contract  Management  Agency  agreed  with  one 
recommendation  and  partially  agreed  with  the  second.  The 


Defense  Contract  Management  Agency  stated  that  it  would 
update  the  memorandum  of  agreement  between  the  Defense 
Contract  Management  Agency  and  the  Joint  Program  Office, 
regarding  surveillance;  however,  we  disagree  and  desire 
specifics  on  the  level  of  oversight  at  contractor  facilities. 

The  following  table  identifies  recommendations  requiring 
an  additional  comment  by  the  Joint  Program  Office  and 
Defense  Contract  Management  Agency.  Please  see  the 
Overall  Findings  and  Recommendations  section  in  the 
report  for  details. 
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Recommendations  Table 


Management 

Recommendations 
Requiring  Comment 

No  Additional 
Comments  Required 

Joint  Program  Office 

A.3,  B.2,  C,  D,  and  E.l.b 

A.l,  A.2,  A.4,  A.5,  B.l,  E.l.a 

Defense  Contract  Management  Agency 

E.2.a  and  b 

*Please  provide  comments  by  October  28,  2013. 


|  DODIG-2013-140  (Project  No.  D2012-DT0TAD-0003.000) 


INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 
4800  MARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA 22350-1 500 


September  30,  2013 


MEMORANDUM  FOR  PROGRAM  EXECUTIVE  OFFICE  JOINT  STRIKE  FIGHTER 

DIRECTOR,  DEFENSE  CONTRACT  MANAGEMENT  AGENCY 


SUBJECT:  Quality  Assurance  Assessment  of  the  F-35  Lightning  II  Program 
(Report  No.  DODIG-2013-140] 

The  DoD  Inspector  General  (IGJ  conducted  a  quality  assurance  assessment  of  the  F-35  Lightning  II 
aircraft  procured  from  Lockheed  Martin.  We  conducted  the  assessment  at  the  Joint  Program  Office  (JPOJ, 
onsite  at  Lockheed  Martin,  and  at  the  major  subcontractors  during  FYs  2012  and  2013.  Our  objective  was 
to  assess  quality  assurance  conformity  to  regulatory  and  contractual  requirements  necessary  for 
F-35  aircraft  production. 

Our  assessment  determined  that  the  F-35  JPO  oversight  of  Lockheed  Martin  was  inadequate  and  that  the 
Defense  Contract  Management  Agency  (DCMA)  oversight  of  the  contractors  was  ineffective.  These  issues 
may  result  in  nonconforming  hardware,  less  reliable  aircraft,  and  increased  cost.  Throughout  the  assessment, 
we  issued  Notices  of  Concerns  to  the  F-35  JPO  to  ensure  timely  corrective  action  of  our  findings.  It  is  our 
understanding  that  the  JPO  has  been  implementing  corrective  actions  and  the  DoD  IG  will  perform  future 
assessments  of  this  critical  program. 

The  draft  version  of  our  report  made  recommendations  to  the  F-35  JPO  and  DCMA  to  ensure  compliance 
with  quality  management  standards  throughout  the  F-  35  supply  chain  and  to  assess  the  impacts  and  risks 
to  all  delivered  aircraft.  We  considered  management  comments  on  the  draft  from  JPO  and  DCMA. 
We  request  further  comments  from  the  JPO  on  Recommendations  A.3,  B.2,  C,  D,  and  E.l.b  and  from  DCMA  on 
Recommendations  E.2.a  and  E.2.b.  Further  comments  should  be  received  by  October  28,  2013. 

DoD  Directive  7650.3  requires  that  recommendations  be  resolved  promptly.  If  possible,  send  a  .pdf  file 
containing  your  comments  to  alois.dopita@dodig.mil.  Copies  of  your  comments  must  have  the  actual  signature 
of  the  authorizing  official  for  your  organization.  We  are  unable  to  accept  the  /Signed/  symbol  in  place  of 
the  actual  signature.  If  you  arrange  to  send  classified  comments  electronically,  you  must  send  them  over  the 
SECRET  Internet  Protocol  Router  Network  (SIPRNETJ. 

We  appreciate  the  courtesies  extended  to  our  staff.  Please  direct  questions  to  Mr.  A1  Dopita  at  [703]  699-0220  or 

alois.dopita@dodig.mil. 


■Randolph  R.  Stone 
Deputy  Inspector  General 
Policy  and  Oversight 


cc:  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 
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Introduction 


Objectives 

Our  objective  was  to  assess  the  F-35  Lightning  II  Program  (F-35  Program],  Joint 
Program  Office  (JPO],  the  prime  contractor,  and  major  subcontractors  conformity 
to  the  contractually  required  Aerospace  Standard  (ASJ9100,  "Quality  Management 
Systems  -  Requirements  for  Aviation,  Space  and  Defense  Organizations,”  contractual 
quality  assurance  clauses,  and  internal  quality  assurance  processes  and  procedures. 


Background 

The  F-35  Program  is  a  joint,  multinational  acquisition  to  develop  and  field  an  affordable, 
next-generation  strike  fighter  aircraft  for  the  Navy,  Air  Force,  Marine  Corps,  and  eight 
international  partners:  the  United  Kingdom,  Italy,  the  Netherlands,  Turkey,  Canada, 
Australia,  Denmark,  and  Norway.  The  F-35  has  three  variants.  The  Conventional 
Takeoff  and  Landing  (CTOL],  Short  Takeoff  and  Vertical  Landing  (STOVL],  and  Carrier- 
Suitable  Variant  (CV]. 


Lockheed  Martin  Aeronautics  Company  (Lockheed  Martin]  entered  system  development 
and  demonstration  in  October  26,  2001.  Lockheed  Martin  has  two  principal 
subcontractors/suppliers,  Northrop  Grumman  Aerospace  Systems  (Northrop  Grumman] 
and  BAE  Systems  (BAE],  Figure  1  shows  the  breakdown  of  the  manufacturing  for  major 
assemblies  between  Lockheed  Martin  and  the  principal  subcontractors.  The  program 
has  about  1,300  other  suppliers,  with  production  occurring  in  47  states  and  Puerto  Rico. 
Additionally,  production  is  occurring  in  more  than  600  suppliers  in  30  other  countries. 


Figure  1.  Manufacturing  Breakdown  of  F-35  Major  Assemblies 
Source:  Image  courtesy  of  JPO 
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According  to  the  Government  Accountability  Office  reports,  JPO  rebaselined  the  F-35 
Program  in  2004  following  weight  and  performance  problems  and  rebaselined  again  in 
2007  because  of  additional  cost  growth  and  schedule  delays.  In  March  2010,  JPO  declared 
that  the  program  exceeded  critical  cost  growth  thresholds  established  by  statute — a 
condition  known  as  a  Nunn-McCurdy  breach.  JPO  continued  extensive  restructuring 
actions  during  2011  and  2012  that  added  more  funding,  extended  schedules,  and  further 
reduced  aircraft  procurement  quantities  in  the  near-term.  The  quantity  of  F-35  aircraft 
to  be  procured  was  not  changed,  but  restructured  plans  deferred  the  procurement 
of  410  aircraft  until  2017.  In  March  2012,  JPO  established  a  new  acquisition  program 
baseline  for  the  F-35  program,  which  incorporated  all  program  restructuring  actions.  The 
March  2012  baseline  represented  the  fourth  rebaseline  since  the  program's  inception. 

According  to  the  latest  acquisition  strategy,  the  F-35  Program  is  one  of  concurrent 
development,  production,  and  sustainment  with  nine  separate  low-rate  initial  production 
(LRIP]1  deliveries.  Each  LRIP  represents  an  increasing  level  of  maturity  as  additional 
system  capability  is  delivered.  Each  LRIP  also  represents  its  own  contract  that  establishes 
the  number  of  aircraft  to  be  produced  and  its  own  acquisition  approach.  For  LRIPs 
1  through  3,  JPO’s  acquisition  approach  was  cost-plus  incentive/award/fixed  fee,  but 
for  LRIPs  4  through  9  a  fixed-price  incentive  (firm  target]  type  contract  is  being  used. 
The  contract  fee  methodologies  also  differ  for  the  various  efforts  on  each  contract  with 
incentive  fee  for  cost  and  schedule,  award  fee  for  timeliness  and  quality,  and  fixed  fee  for 
diminishing  manufacturing  sources. 

Quality  Trend  Data 

F-35  Program  quality  metric  data  show  improvement  in  scrap,  rework,  and  repair  rates  and 
in  software  and  hardware  quality  action  requests  per  aircraft.  However,  the  Government 
incurred  and  will  continue  to  incur  a  significant  cost  for  these  issues,  either  through  the 
previous  cost-plus  incentive/award/fixed-fee  contracts  or  via  quality  incentives  onfuture 
fixed-price  incentive-fee  contracts.  As  of  March  2013,  metric  data  showed  that  there 
were,  on  average,  972  quality  action  requests  per  aircraft  for  LRIP  1,  987  for  LRIP  2,  926 
for  LRIP  3,  and  859  for  LRIP  4.  Scrap,  rework,  and  repair  rates  on  average  per  aircraft 
were  13.82  percent  for  FY  2012  and  13.11  percent  for  FY  2013,  thus  showing  only  a 
moderate  change  towards  reducing  costs.  Although  it  would  be  unrealistic  to  expect  first 
production  to  be  issue  free,  our  contractor  assessments  indicate  that  greater  emphasis 
on  quality  assurance,  requirement  flow  down,  and  process  discipline  is  necessary,  if  the 
Government  is  to  attain  lower  program  costs. 


1  LRIP  as  defined  by  the  Defense  Acquisition  University  is  the  "effort  intended  to  result  in  completion  of  manufacturing 
development  in  order  to  ensure  adequate  and  efficient  manufacturing  capability  and  to  produce  the  minimum  quantity 
necessary  to  provide  production  or  production-representative  articles  for  Initial  Operational  Test  and  Evaluation." 
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Assessment  Criteria 

AS9100  Standard 

The  F-35  LRIP  contracts  require  that  the  contractor  comply  with  AS9100.  We  performed 
our  assessments  to  the  AS9100C  standard  because  it  was  the  current  version  at  the  time 
of  the  assessments,  and  any  defense  contractor  obtaining  certification  would  be  required 
to  meet  that  standard. 

The  AS9100C  standard  breaks  down  quality  assurance  requirements  into  five 
major  clauses: 

•  Quality  Management  System, 

•  Management  Responsibility, 

•  Resource  Management, 

•  Product  Realization,  and 

•  Measurement,  Analysis,  and  Improvement. 

The  Quality  Management  System,  Management  Responsibility,  and  Resource 
Management  clauses  require  the  organization  to  have  a  quality  assurance  management 
organization  that  has  all  the  resources  and  authority  to  affect  the  end-item  quality  of 
the  product.  In  addition,  it  requires  the  organization  to  have  a  quality  assurance  manual 
and  strict  controls  over  all  documentation,  data,  and  procedures  that  affect  the  quality 
of  the  product.  Product  Realization  covers  the  activities  and  processes  necessary  to 
bring  a  product  into  existence. 

Product  realization  is  broken  down  further  in  AS9100  as  follows: 

•  Planning  of  Product  Realization, 

•  Customer-Related  Processes, 

•  Design  and  Development, 

•  Purchasing, 

•  Production  and  Service  Provision,  and 

•  Control  of  Monitoring  and  Measuring  Equipment. 

Planning  of  Product  Realization  requires  the  organization  to  develop  processes  needed 
for  design  and  development  of  product  and  includes  elements  such  as  procedures, 
quality  assurance  records,  resource  requirements,  safety  and  reliability  programs,  and 
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inspection  and  test.  Design  and  Development  includes  requirements  that  cover  planning, 
inputs,  outputs,  review,  verification,  validation,  and  control  of  changes  as  related  to 
design  and  development.  Purchasing  requires  the  organization  to  ensure  that  the 
purchased  product  conforms  to  specified  purchase  requirements  and  that  all  products 
purchased  from  suppliers  are  verified  against  purchase  agreement  requirements.  The 
Production  and  Service  Provision  requires  the  organization  to  ensure  that  production  is 
accomplished  under  controlled  conditions  using  drawings  and  specifications,  work 
instructions,  production  tools  and  software  programs,  monitoring  and  measuring 
equipment,  and  evidence  that  all  production  and  inspection/verification  operations  have 
been  completed  as  planned. 

Measurement,  Analysis,  and  Improvement  requires  the  organization  to  ensure  the 
product  continuously  improves.  The  clause  includes  customer  satisfaction,  internal 
audit,  monitoring  and  measuring  processes  and  product,  and  control  of  nonconforming 
products  to  ensure  continual  improvement. 

Aviation  Critical  Safety  Items  Requirements 

In  addition  to  AS9100,  we  assessed  the  JPO  and  the  contractor's  implementation  of 
aviation  critical  safety  items  (CSIs]  requirements.  A  CSI  is  a  part,  assembly,  or  support 
equipment  whose  failure  could  cause  loss  of  life,  permanent  disability  or  major  injury, 
loss  of  a  system,  or  significant  equipment  damage.  Special  attention  should  be  paid  to 
CSIs  to  prevent  the  potential  catastrophic  or  critical  consequences  of  failure.  CSIs  require 
special  handling,  engineering,  manufacturing,  and  inspection  documentation  to  control 
and  ensure  safety  of  flight. 

Public  Law  108-136,  Section  802,  “Quality  control  in  procurement  of  aviation  CSIs  and 
related  services,"  requires  DoD  to  prescribe  a  quality  control  policy  for  the  procurement 
of  aviation  CSIs.  Joint  Service  CSI  Instruction,  “Management  of  Aviation  Critical  Safety 
Items,"  implements  the  DoD  CSI  program  and  establishes  the  policies,  procedures,  and 
responsibilities  to  manage  CSI.  The  Joint  Aeronautical  Commanders  Group  (JACGJ,  “JACG 
Aviation  Critical  Safety  Item  Handbook,"  implements  the  policies  in  the  Joint  Service  CSI 
Instruction  and  describes  the  technical  and  quality  assurance  requirements  for  a  Prime 
Contractor  CSI  program. 

Statutory  and  Regulatory  Requirements 

Additionally,  we  assessed  the  compliance  of  applicable  statutory  and  regulatory 
requirements  to  include  requirements  in  the  Federal  Acquisition  Regulation; 
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DoD  Instruction  5000.02,  "Operation  of  the  Defense  Acquisition  System,"  Enclosure  4;  and 
DoD  4140. 1-R, "  DoD  Supply  Chain  Materiel  Management  Regulation,"  Section  C3.6. 

Quality  Assurance  Assessment  Process 

To  evaluate  the  JPO's  management  of  the  F-35  quality  assurance  program,  we  performed 
a  series  of  quality  assurance  assessments  of  JPO,  prime  contractor,  and  major 
subcontractors.  We  assessed  conformity  to  the  contractually  required,  AS9100, 
contractual  quality  assurance  clauses,  internal  quality  assurance  processes,  and 
procedures  for  the  following  six  contractors: 

•  Lockheed  Martin  Aerospace  Company,  Fort  Worth,  Texas  (Prime  Contractor 
and  Aircraft  Integrator]; 

•  Northrop  Grumman  Aerospace  Systems,  El  Segundo  and  Palmdale,  California 
(Center  Fuselage  Integrator]; 

•  BAE,  Samlesbury,  United  Kingdom  (Aft  Fuselage  Integrator]; 

•  L-3  Display  Systems  (L-3],  Alpharetta,  Georgia  (Panoramic  Cockpit 
Display  System]; 

•  Honeywell  Aerospace  (Honeywell],  Yeovil,  United  Kingdom  (On-Board  Oxygen 
Generation  System];  and 

•  United  Technologies  Corporation,  Aerospace  Systems  (UTAS],  Fort  Worth, 
Texas  and  Independence,  Ohio  (Landing  Gear  System], 

We  selected  the  contractors  based  on  product  criticality  and  risk.  For  each  assessment, 
we  established  teams  of  engineering  and  subject  matter  experts  who  assessed  to  the 
AS9100C  Quality  Management  System  standard.  The  subject  matter  expert  teams 
consisted  of  14  to  18  quality  assurance  engineers,  trained  and  certified  in  AS9100,  who 
had  an  average  of  15  years  of  quality  assurance  audit  experience.  Additionally,  at  the 
Lockheed  Martin,  Northrop  Grumman,  and  BAE  assessments,  we  included  a  team  that 
evaluated  the  aviation  CSI  process. 

This  assessment  focused  on  quality  management  system  compliance.  We  did  not 
examine  whether  financial  restitution  was  granted  for  nonconformances,  variances, 
waivers,  deviations,  etc. 
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Notice  of  Concern 

At  the  conclusion  of  each  contractor  assessment,  we  issued  a  notice  of  concern  (NOC]  to 
the  JPO  to  ensure  timely  corrective  action  of  each  finding.  The  NOCs  included  the  detailed 
findings  from  that  location  to  ensure  prompt  resolution. 

Classification  and  Categorization  of  Findings 

We  wrote  363  findings  that  identified  a  total  of  719  issues  for  the  six2  contractor 
assessments  performed.  There  were  multiple  issues  identified  in  most  of  the  findings 
with  the  majority  of  issues  were  violations  of  the  AS9100C  Quality  Management 
System  standard.  For  each  of  the  assessments,  we  classified  the  findings  as  major 
nonconformances,  minor  nonconformances,  or  opportunities  for  improvement  (OFIs], 
Each  finding  received  an  additional  technical  review  for  accuracy  and  classification. 

As  defined  by  the  AS9101  standard,  a  major  nonconformance  is  a  nonfulfillment  of  a 
requirement  that  is  likely  to  result  in  the  failure  of  the  quality  management  system  or 
reduce  its  ability  to  ensure  controlled  processes  or  compliant  products/services.  A 
minor  nonconformance  is  a  nonfulfillment  of  a  requirement  that  is  not  likely  to  result  in 
the  failure  of  the  quality  management  system  or  reduce  its  ability  to  ensure  controlled 
processes  or  compliant  products  or  services.  An  0F1  is  an  industry  best  practice  where 
a  specific  requirement  does  not  exist.  This  report  focuses  on  the  major  findings  for  each 
respective  contractor. 

The  table  shows  the  breakdown  of  major,  minor,  and  OFI  findings  for  each  of  the 
site  assessments. 


Table.  Major/Minor/OFI  Finding  Totals  at  Each  Site* 


Location 

Major 

Minor 

OFI 

Lockheed  Martin 

28 

42 

0 

Northrop  Grumman 

23 

42 

1 

BAE 

36 

44 

2 

L-3  Communications 

25 

30 

1 

Honeywell 

18 

20 

0 

UTAS 

17 

34 

0 

Total 

147 

212 

4 

*Findings  include  those  written  against  JPO  and  DCMA  during  the  respective  assessments. 


2  We  conducted  a  pre-assessment  visit  at  Honeywell,  Phoenix,  Arizona.  Although  the  site  was  not  selected  for  assessment, 
we  wrote  one  finding  against  the  Defense  Contract  Management  Agency. 
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For  each  of  the  contractor  assessments,  we  categorized  the  findings  by  the 
relevant  AS9100C  clause.  Figure  2  provides  the  breakdown  of  these  findings  for  the 
overall  assessment 


Figure  2.  Categorization  of  Findings 
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Contractor  Assessments 

Lockheed  Martin  Aeronautics  Company 
(Prime  Contractor) 

(Fort  Worth,  Texas] 

Lockheed  Martin  is  the  prime  contractor  and  lead  integrator  for  the  F-35  Program. 
Lockheed  Martin  manufactures  the  forward  fuselage  and  wings  and  performs  final 
integration  and  final  aircraft  verification  at  the  Fort  Worth,  Texas,  facility  (Figure  3], 

Our  assessment  of  Lockheed  Martin  resulted  in  70  findings  that  identified  weaknesses 
in  Lockheed  Martin's  implementation  of  an  AS9100  Quality  Management  System.  Many 
of  the  issues  documented  in  the  findings  indicate  additional  F-35  Program  risks  that 
could  impact  cost,  schedule,  and  performance.  We  wrote  an  additional  24  findings 
against  Lockheed  Martin  stemming  from  our  assessments  at  its  subcontractor  sites 
(7  at  L-3  Communications,  5  at  Northrop  Grumman,  7  at  BAE,  and  5  at  UTAS],  Figure  4 
provides  the  Lockheed  Martin  findings  by  AS9100  clause  with  the  following  sections 
summarizing  significant  issues  documented  during  the  assessment. 
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Figure  4.  Lockheed  Martin  Findings 
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Documentation  Requirements  (4.2) 

We  wrote  several  findings  regarding  documentation  control.  For  example,  the  material 
data  system  that  automatically  records  the  cumulative  product  excursion  time3  of 
composites  prior  to  curing  requires  corrective  action.  When  products  required  a 
secondary  lay-up,  the  excursion  time  continued  to  accumulate  in  the  material  data 
system,  even  though  the  product  cured.  To  preclude  the  appearance  that  the  materiel 
had  exceeded  its  useful  life,  operators  were  overriding  the  automated  materiel  data 
system  with  manual  entries.  We  could  not  determine  if  these  manual  entries  accurately 
reflected  the  actual  excursion  time  of  the  material.  The  ability  of  operators  to  override 
the  recorded  excursion  time  data  prevents  the  data  record  from  reflecting  whether 
the  final  product  conformed  to  requirements. 


3 


The  excursion  time  reflects  the  amount  of  time  available  for  the  operator  to  use  the  material,  typically  for  bonding  agents. 
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Other  findings  documented  similar  record  control  deficiencies  including  maintenance  of 
flight  line  security  logs  and  unclear  record  retention  requirements.  In  addition,  several 
procedures,  checklists,  and  program  plans  were  in  use  prior  to  approval  and  formal 
release.  Lockheed  Martin  personnel  were  using  unapproved  (without  engineering  and 
quality  assurance  approval]  procedures  for  18  months.  The  unapproved  procedures 
were  used  to  control  the  vacuum  hold-down  fixtures  for  both  the  flexible  overhead 
gantry  machining  and  coordinate  measurement  machine.  In  addition,  the  manufacturing 
plans  for  the  wing  systems  and  wing  structures  were  labeled  as  "uncontrolled/for 
reference  only,”  although  they  were  used  by  the  planning  department  to  develop  the 
work  instruction  for  aircraft  assembly.  Maintaining  accurate  product  records  and 
controlling  process  documentation  is  necessary  to  ensure  that  the  product  meets 
engineering  and  customer  requirements. 

Human  Resources  (6.2) 

Competence,  Training,  and  Awareness  (6.2.2).  We  found  Lockheed  Martin's 
management  had  not  updated  the  employee  training  requirement  to  reflect  actual 
training  needs.  The  Learning  Management  System  included  80  employees  with  expired 
certifications  such  as  ejection  seat  installation  and  removal,  F-35  egress  system  safety, 
and  explosives  care,  handling,  and  storage.  In  addition,  employee  training  plans 
identified  training  certifications  that  were  not  required  for  the  jobs  those  employees 
were  performing.  In  another  example,  37  operators  were  working  on  flight  hardware 
in  a  foreign  object  debris/damage  (FOD)-control  area  without  current  certifications. 
A  subsequent  major  finding,  noted  later  in  this  report  (see  paragraph  7.5.5),  documented 
that  FOD  was  discovered  in  FOD-critical  and  FOD-control  areas.  Training  ensures 
operators  are  cognizant  of  the  latest  industry  standards  and  techniques;  it  also  provides 
increased  awareness.  The  lack  of  management  attention  to  certification  requirements  of 
employees  working  in  aircraft  assembly  areas  places  the  delivered  product  at  risk. 

Planning  of  Product  Realization  (7.1) 

We  identified  several  major  findings  in  Lockheed  Martin's  Planning  of  Product 
Realization.  A  major  finding  noted  that  Lockheed  Martin's  shop  floor  planning  contained 
incorrect  verification  steps  that  resulted  in  verifications  either  not  being  completed  or 
unnecessary.  For  example,  the  center  wing  mate  assembly  work  instruction  required 
torque  verification  for  fasteners;  however,  fasteners  involved  in  this  operation  did  not 
require  torqueing.  In  another  instance,  the  wing  systems  installation  work  instructions 
required  performing  two  verifications;  one  for  verification  of  the  electrical  bond  and 
the  other  for  electrical  resistance.  A  review  of  drawing  requirements  and  discussions 
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with  personnel  noted  that  the  operators  were  not  required  to  perform  the  electrical 
resistance  verification,  even  though  it  was  later  determined  to  be  required.  Finally,  the 
wing  system  installation  work  instructions  identified  two  different  inspection  criteria  for 
electrical  bond  resistance  inspection;  however,  it  did  not  specify  which  one  applied. 

Another  major  finding  noted  that  manufacturing  documentation  did  not  identify  an 
inspection  requirement  that  would  verify  that  sealing  of  fasteners  on  skin  installations  met 
dimensional  requirements.  Specifically,  the  drawing  specified  minimum  fillet  dimensions 
for  integral  fuel  tank  sealing;  however,  the  dimensions  were  only  inspected  visually  and 
were  not  measured.  Sealing  adds  weight  to  the  aircraft  and  because  measurements  were 
not  taken,  it  is  unclear  whether  this  would  be  accounted  for  in  final  system  performance. 

Planning  and  procedures  in  several  process  areas  lacked  sufficient  detail  to  provide 
adequate  work  instructions.  For  example,  in  the  metal  machining  areas,  the  procedures 
did  not  identify  details  regarding  part  cleaning,  setup,  and  mechanical  clamping;  and 
in  the  wing  assembly  area,  the  fillet  sealing  tools  were  not  identified  in  the  procedure, 
and  personnel  were  using  unspecified  tools.  Also,  a  specification  for  the  integral  fuel 
tank  sealing  had  numerous  errors  in  the  graphics  depicting  dimensional  identification 
along  with  missing,  incomplete,  or  incorrect  dimensional  lines.  In  the  wing  box 
integration  area  of  the  facility,  the  planning  did  not  detail  numerous  operations  for 
skin  panel  cover  closeout.  This  lack  of  precise  and  accurate  work  instructions  could 
result  in  nonconformances  later  in  processing  or  after  fielding,  thus  causing  schedule 
delays  and  additional  program  costs. 

A  major  finding  noted  that  Lockheed  Martin  was  not  implementing  its  diminishing 
manufacturing  sources  and  materiel  shortages  [DMSMS]  process  in  accordance 
with  internal  procedures.  For  example,  Lockheed  Martin  did  not  always  include  the 
requirement  for  the  delivery  of  bills  of  materials  in  its  supplier  statement  of  work  (SOW] 
and  did  not  always  upload  subtier  supplier  bills  of  materials  into  its  obsolescence¬ 
forecasting  tool.  This  precluded  Lockheed  Martin  from  proactively  managing  DMSMS. 
Cost  metrics  provided  by  Lockheed  Martin  indicate  that  the  cost  of  DMSMS  has  been 
increasing.  Proactive  management  of  DMSMS  is  critical  for  containing  program  cost. 

Lockheed  Martin  had  not  assessed  the  use  of  lead-free  electronic  parts  throughout  the 
F-35  Program  as  required  by  its  own  corporate  requirements,  "Lockheed  Martin 
Corporation  Lead-Free  Control  Plan."  Lockheed  Martin  did  not  flow  down  a  lead-free 
control  plan  requirement  to  its  suppliers.  The  use  of  lead-free  electronic  parts  in  critical 
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applications  increases  the  risk  to  component  service  life,  reliability,  airworthiness,  and 
safety  because  of  tin  whisker  growth.4 

A  final  major  finding  identified  that  Lockheed  Martin  did  not  always  define  the  capability 
confirmation  criteria  used  for  acceptance  of  aircraft.  In  some  cases,  confirmation  criteria 
did  not  include  clearly  measurable  and  verifiable  acceptance  criteria,  or  were  not  traceable 
to  source  data  necessary  to  confirm  the  required  capability.  As  a  result,  acceptance  of 
F-35  aircraft  depends  on  the  judgment  of  individual  subject  matter  experts  without  the 
use  of  clearly  measurable  and  verifiable  acceptance  criteria  to  confirm  the  capability  of 
the  aircraft.  Unclear  capability  confirmation  criteria  in  the  configuration  and  capability 
description  documents  can  lead  to  inconsistencies  or  quality  escapes5  during  acceptance 
of  LRIP  aircraft. 

Configuration  Management  (7.1.3).  Our  assessment  identified  that  Lockheed 
Martin  neither  adequately  provided  review  or  approved  of  engineering  change 
submittals  made  by  Lockheed  Martin's  critical  suppliers.  Specifically,  Lockheed  Martin 
neither  approved  or  disapproved  all  L-3  major  changes  nor  provided  concurrence  in 
classification  for  all  minor  changes  submitted  by  L-3  and  UTAS.  L-3  submitted  8  major 
engineering  change  proposals  to  Lockheed  Martin  since  May  7,  2007,  with  no  evidence 
of  approval  or  disapproval,  and  14  minor  changes  through  engineering  change  notices 
during  2011,  with  no  response  for  "concurrence  in  classification."  This  indicates  a 
breakdown  in  a  basic  quality  assurance  process  used  to  ensure  that  Lockheed  Martin's 
systems  engineering  understand  and  agree  or  disagree  with  supplier  product  changes. 
Another  major  finding  noted  that  Lockheed  Martin  did  not  require  L-3  to  perform 
configuration  status  accounting  for  LRIP  1  through  5  delivered  aircraft.  Failure  to  require 
suppliers  to  perform  configuration  status  accounting  of  delivered  hardware  can  result 
in  the  inability  to  provide  hardware  traceability  in  support  of  failure  analysis,  system 
upgrades,  and  supportability  of  the  aircraft.  Lockheed  Martin's  configuration  management 
processes  did  not  define  configuration  management  board  membership  and  authorities. 
In  addition,  the  configuration  management  plan,  imposed  on  BAE  by  Lockheed  Martin, 
had  invalid  references  and  processes  for  BAE  to  follow. 

Design  and  Development  (7.3) 

Design  and  Development  Planning  (7.3.1).  A  major  finding  identified  that  Lockheed 
Martin  was  delivering  F-35  aircraft  that  did  not  meet  required  interchangeability- 


4  A  phenomenon  known  as  "tin  whiskers”  becomes  a  factor  due  to  the  absence  of  lead  in  tin-bearing  solder  and  component 
finishes.  Tin  whiskers  can  cause  electrical  shorts  resulting  in  system  failures. 

5  A  quality  escape  is  nonconforming  material  that  has  entered  the  product,  supply  chain,  or  proceeded  beyond  the 
acceptance  process. 
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replaceability  contract  requirements  dating  back  to  2001.  This  deficiency  was  identified 
in  May  2009  by  the  Defense  Contract  Management  Agency  (DCMA],  Lockheed  Martin 
identified  this  as  a  high-risk  item  and  developed  a  35-step  plan  for  mitigation;  however, 
final  mitigation  is  not  expected  to  be  complete  until  2015.  The  interchangeability- 
replaceability  plan  was  to  validate  through  demonstration  277  interchangeability- 
replaceability  components  that  were  primarily  comprised  of  aircraft  skin  assemblies.  At 
the  time  of  our  assessment,  273  demonstrations  were  still  required  and  it  was  estimated 
that  150  aircraft  would  be  fielded  before  completing  the  plan.  This  will  inevitably 
create  fielded  aircraft  components  that  are  not  interchangeable  or  additional  cost  to  the 
Government  for  bringing  fielded  aircraft  up  to  specification. 

A  software  safety  engineer  was  not  assigned  to  the  software  development  integrated 
product  team  (IPT],  as  required  by  the  Software  Development  Plan,  to  ensure  that 
software  meets  all  safety  requirements.  Without  adequate  product  evaluation  of  mission 
system  software,  Lockheed  Martin  cannot  ensure  aircraft  safety  requirements  are  met. 

Lockheed  Martin's  Mission  Systems  Software  Development  Plan  did  not  address  all  the 
requirements  of  failure  mode  testing  specified  in  the  Air  Systems  Software  Development 
Plan.  Additionally,  the  Test  Readiness  Review  for  air  system  software  block  2A  did  not 
contain  exit  criteria,  as  required  by  the  Systems  Engineering  Plan.  If  requirements  are 
not  specified  in  the  program  plans,  and  technical  reviews  do  not  contain  exit  criteria,  the 
aircraft  design  may  not  meet  all  program  requirements. 

Lockheed  Martin  was  not  providing  adequate  management  of  the  development  efforts  at 
critical  suppliers.  Multiple  findings  were  written  regarding  Lockheed  Martin's  failure  to 
flow  down  development  planning  requirements.  During  the  UTAS  assessment,  we  found 
that  Lockheed  Martin  flowed  down  the  F-35  Brake/Skid  Control  System  performance- 
based  specification  with  incomplete  and  conflicting  specification  requirements  and  did 
not  flow  down  required  safety  and  mission-critical  function  information.  Additionally, 
Lockheed  Martin  provided  incomplete,  conflicting,  and  ambiguous  system  requirements 
in  the  Panoramic  Cockpit  Display  performance  specification  to  L-3. 

During  the  Northrop  Grumman  assessment,  we  identified  that  Lockheed  Martin  did  not 
maintain  mission  systems  requirements  traceability  to  the  software-level  requirements. 
Specifically,  the  requirements  were  not  derived  from  or  traceable  to  the  allocated 
software  requirements  and  the  top-level  system  requirements.  Untraceable  requirements 
cannot  be  verified  for  impact  on  system  performance.  In  addition,  Lockheed  Martin  did 
not  maintain  the  air  system  block  plan  to  reflect  current  capabilities  planned  for  the 
F-35  Program  software  blocks.  The  plan  had  not  been  updated  since  August  2008  and  did 
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not  reflect  current  block  planning,  which  would  define  air  system  capabilities.  As  a  result, 
there  is  no  authoritative  document  defining  current  and  planned  software  capabilities. 
Without  a  current  air  systems  block  plan,  software  program  capabilities  may  not  be 
implemented  when  required. 

During  the  BAE  assessment,  we  identified  that  Lockheed  Martin  was  not  ensuring  that  BAE 
was  working  to  the  current  revision  of  the  software  development  plan  for  the  F-35  mission 
systems  software.  Software  products  developed  by  BAE  using  outdated  mission  systems 
requirements  may  not  contain  current  development  processes  and  acceptance  criteria. 
In  addition,  several  of  the  F-35  subsystem,  software  development  plans  did  not  include 
software  maintenance,  even  though  the  software  was  already  in  the  maintenance  phase. 

For  some  critical  design  reviews  (CDRs],  Lockheed  Martin  did  not  establish  or  require 
exit  criteria.  Additionally,  Lockheed  Martin  did  not  formally  close  all  design  review  action 
items  and  did  not  disposition  or  approve  the  UTAS  failure  modes,  effects,  and  criticality 
analysis  (FMECA]  reports  for  the  main  and  nose  landing  gear. 

The  lack  of  part  interchangeability,  insufficient  requirement  flow  down,  and  open  design 
review  action  items,  as  well  as  failing  to  evaluate  and  approve  or  disapprove  engineering 
deliverables  represent  product  instability  and  risk. 

Purchasing  (7.4) 

Our  assessment  documented  several  findings  citing  inadequacies  in  Lockheed  Martin's 
oversight  of  its  suppliers  and  management  of  subcontractor  deliverables.  For  example, 
a  major  finding  noted  that  Lockheed  Martin  did  not  manage  subcontractor  deliverables 
in  accordance  with  the  approved  process.  Many  of  the  deliverables  identified  in  the 
subcontract  SOW  and  associated  subcontractor  deliverable  requirements  lists  had  not 
been  delivered  or  were  delivered  more  than  2  years  late  without  follow-up  by  Lockheed 
Martin.  Several  of  these  deliverables;  such  as  the  software  development  plan,  configuration 
management  process  plan,  manufacturing  plan,  electronic  bill  of  materials,  advanced 
quality  assurance  plan,  and  acceptance  test  plan;  still  require  concurrence  or  approval 
by  Lockheed  Martin.  Additionally,  the  data  management  system  for  tracking  the  status  of 
supplier  deliverables  contained  inconsistent  information  with  requirements  contained  in 
the  subcontract  deliverable  requirements  lists. 

Several  other  findings  noted  inconsistencies  in  the  supplier  management  and  control 
process.  For  example,  Lockheed  Martin  did  not  always  conduct  supplier  evaluations  as 
required.  Additionally,  the  evaluation  process  for  software  suppliers  did  not  adequately 
identify  requirements  for  supplier  approval  and  control,  such  as  a  detailed  supplier 
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approval  process,  detailed  oversight  strategy,  supplier  risk  levels,  and  the  periodicity  for 
supplier  selection  audits  or  in-process  supplier  audits.  These  evaluations  are  a  primary 
input  into  the  supplier  rating  program.  In  another  finding,  several  suppliers  were  issued 
corrective  action  requests;  however,  these  suppliers  were  delinquent  in  responding  to 
the  requests  and  Lockheed  Martin  did  not  follow-up  to  determine  the  status.  Lockheed 
Martin  cannot  ensure  suppliers  are  meeting  program  requirements  without  a  robust 
supplier  management  system. 

Production  and  Service  Provisions  (7.5) 

Control  of  Production  and  Service  Provision  (7.5.1).  Lockheed  Martin  was  not 
following  established  procedures  to  maintain  production  control.  For  example,  several 
mechanics  were  violating  procedures  by  not  wearing  gloves  while  installing  the  fasteners 
for  the  wing  close-up  operation.  These  particular  fasteners  have  a  chemical  film  dry  lube 
coating  on  the  threads  and  require  gloves  to  avoid  contamination  during  installation.  In 
addition,  an  operator  performing  the  nutplate  push  test  for  the  CTOL  wing  was  not  pushing 
the  required  amount  of  time  at  each  test  point  resulting  in  uncertainty  that  proper  epoxy 
adhesion  was  obtained. 

Two  findings  documented  a  lack  of  adherence  to  procedures  regarding  rework  and  repair 
documentation.  Lockheed  Martin  performed  rework  on  the  Rear  Spar  Brackets  without 
issuing  a  quality  assurance  report.  Another  finding  identified  undocumented  damage  to 
the  aluminum  shims  of  an  aircraft  assembly.  Assembly  personnel  notified  the  supervisor 
of  the  approximate  3x12  inch  damaged  area  to  the  aluminum  shims;  however,  they 
did  not  write  a  nonconformance  report  in  compliance  with  Lockheed  Martin  process. 
Nonconforming  product  should  be  identified  and  dispositioned  by  the  appropriate 
engineering  expertise  to  ensure  sufficient  repair  and  eliminate  future  occurrences.  As  a 
result,  a  serious  quality  escape  could  occur,  affecting  reliability  or  safety  of  flight. 

Production  Process  Verification  (7.5. 1.1).  Several  First  Article  Inspection  (FAI)6 
findings  were  documented  against  Lockheed  Martin.  The  purpose  of  FAI  is  to  obtain  a 
representative  production  sample  and  ensure  that  all  key  design  characteristics  have 
been  achieved,  critical  processes  are  controlled,  and  unit-to-unit  repeatability  is  attained. 

A  major  finding  documented  several  deficiencies  with  Lockheed  Martin's  application  of 
the  FAI  process.  Specifically,  Lockheed  Martin  applied  AS9102  guideline  requirements 
inconsistently.  AS9102  establishes  the  requirements  for  performing  and  documenting 


6  AS9102  defines  FAI  as,  "A  complete,  independent,  and  documented  physical  and  functional  inspection  process  to  verify 
that  prescribed  production  methods  have  produced  an  acceptable  item  as  specified  by  engineering  drawings,  planning, 
purchase  order,  engineering  specifications,  and/or  other  applicable  design  documents." 
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FAIs.  Lockheed  Martin  imposed  this  standard  on  all  suppliers  of  F-35  critical  assemblies; 
however,  it  did  not  require  conformance  to  AS9102  on  a  product  fabricated  within  its  own 
facility.  In  addition,  Lockheed  Martin's  FAI  process  focused  primarily  on  a  100-percent 
inspection  of  products  that  met  engineering  requirements  and  did  not  include  verification 
of  process  stability  and  personnel  proficiency.  These  tenets  are  required  by  AS9102,  on 
which  Lockheed  Martin  based  its  FAI  plan.  Finally,  AS9102  does  not  exclude  assemblies 
from  FAIs,  but  Lockheed  Martin  excluded  FAIs  on  major  assemblies  such  as  the  wings, 
tail,  forward,  center,  and  aft  fuselage  sections  of  the  aircraft.  Many  process  changes 
at  Lockheed  Martin  were  occurring  in  the  major  assembly  areas  and  involved  moving 
tooling  and  equipment  to  perform  out-of-station  work.  According  to  AS9102,  changes 
in  manufacturing  processes,  location  of  manufacture,  tooling,  or  materials  provide  the 
rationale  for  performing  or  reperforming  full  or  partial  FAIs.  Overall,  Lockheed  Martin's 
FAI  process  was  ineffective,  as  evidenced  by  the  numerous  planning,  tooling,  and 
training  deficiencies  documented. 

We  found  that  Lockheed  Martin  flowed  down  conflicting  FAI  requirements  to  BAE  and 
Northrop  Grumman.  Specifically,  the  contractually  imposed  AS9102  standard  requires 
that  FAIs  be  performed  on  all  components,  including  assemblies.  However,  Lockheed 
Martin's  FAI  plan  exempts  many  of  the  major  assemblies  and  subassemblies  from  FAIs. 
This  conflict  resulted  in  many  items  not  receiving  FAIs.  Additionally  at  L-3,  Lockheed 
Martin  had  approved  variances  for  L-3  to  ship  display  units  and  electronics  units  to 
Lockheed  Martin  without  completing  FAIs;  another  indicator  of  Lockheed  Martin's 
disregard  for  the  FAI  process. 

Failure  to  adequately  perform  FAIs  could  impact  the  F-35  Program's  ability  to  achieve 
process  stability  and  successfully  meet  program  global  production  rate  goals.  The  lack 
of  process  stability  will  increase  costs  and  schedule  because  of  discrepant  hardware  and 
quality  escapes. 

Control  of  Production  Equipment,  Tools,  and  Software  Programs  (7.5. 1.3).  The 

assessment  identified  a  major  deficiency  with  documenting  the  control  of  manufacturing 
tooling  used  on  the  F-35  Program.  Lockheed  Martin  personnel  were  using  discrepant 
sealant  mixing  equipment  for  production.  Without  engineering  approval  or  analysis, 
Lockheed  Martin  personnel  were  mixing  the  sealant  (for  the  skin  installation  of  the 
left  aft  wingbox)  at  twice  the  required  time  to  compensate  for  the  discrepant  machine. 
Furthermore,  the  compensation  time  used  was  incorrect,  thus  resulting  in  possible  over 
mixing  of  the  sealant.  Other  findings  noted  that  additional  equipment  was  not  maintained, 
including  the  vacuum  system  equipment  used  for  composite  machining  operations, 
another  sealant  mixer,  and  software  for  computer  numerically  controlled  machines. 
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Preservation  of  Product  (7.5.5).  We  documented  major  findings  related  to  control 
of  FOD  and  shelf-life  material.  For  instance,  controls  were  inadequate  to  prevent  FOD 
from  being  introduced  onto  the  production  floor.  A  DoD  Office  of  Inspector  General 
(OIG)  walkthrough  identified  FOD  within  FOD-critical  areas  located  in  the  final  assembly 
production  area  and  flight  line  run  stations.  Examples  include  metal  shavings  found 
throughout  the  left  air  intake  of  the  aircraft,  leaking  fluid  from  a  bagged  hydraulic  line, 
and  a  paper  tag  below  the  aircraft.  FOD  was  also  identified  in  FOD-control  areas  located 
in  the  electronic  mate,  alignment  system,  and  final  assembly  production  areas.  Examples 
include  metal  shavings  found  in  the  cockpit  of  the  aircraft;  and  metal  fasteners,  wood 
fibers,  and  miscellaneous  tools  and  debris  throughout  the  production  area.  Another 
major  finding  documented  several  incidents  where  wiring  harness  protective  connector 
caps  were  missing.  The  caps  are  designed  to  prevent  both  FOD  and  electrostatic  damage 
to  aircraft  components  and  circuitry  during  manufacturing. 

We  found  expired  composite  material  and  a  process  lay-up  kit  with  improperly  identified 
shelf  life  and  excursion  times.  These  items  were  not  impounded  to  prevent  unintended 
use.  Failure  to  properly  identify  and  maintain  traceability  of  shelf-life  material  may  lead 
to  use  of  material  that  does  not  meet  specifications,  potentially  resulting  in  degraded 
material  performance  and  failure. 

Control  of  Nonconforming  Product  (8.3) 

Airframes  measured  using  the  laser  alignment  system  routinely  did  not  meet  mate 
and  alignment  drawing  requirements;  however,  Lockheed  Martin  did  not  identify  that 
the  airframes  were  discrepant.  Numerous  airframes  for  all  three  aircraft  variants  have 
been  integrated  into  end-item  flight  hardware  and  processed  through  final  assembly 
but  Lockheed  Martin  did  not  identify  and  disposition  the  nonconformances.  Two  other 
findings  identified  three  separate  instances  when  "Open"  nonconformances  were  not 
identified  and  subsequently  removed  in  accordance  with  procedures.  The  effective 
identification,  review,  and  disposition  of  nonconforming  product  is  essential  to  prevent 
processing  of  defective  hardware  into  the  aircraft. 

Improvement  (8.5) 

Corrective  Action  (8.5.2).  We  wrote  several  findings  regarding  Lockheed  Martin's 
corrective  action  process.  At  the  L-3  assessment,  Lockheed  Martin  approved  variances 
that  waived  performance  requirements  for  the  electronics  and  display  units.  The 
variances  did  not  contain  documented  corrective  actions  that  would  prevent  the  need  for 
repeated  variance  requests.  There  were  17  performance  variances  on  the  display  unit 
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and  8  performance  variances  on  the  electronic  unit.  L-3  continues  to  produce  and  deliver 
electronics  and  display  units  with  performance  issues  under  these  variances. 

At  the  Northrop  Grumman  assessment,  the  software  tool  (Eagle  Speed]  that  Lockheed 
Martin  provided  to  Northrop  Grumman  did  not  include  identification  and  recording  of 
software  nonconformance  root  cause  as  required  in  the  F-35  mission  systems  software 
development  plan.  This  tool  tracks  and  manages  mission  systems  software  problem 
anomaly  and  software  problem  reports.  Without  identification  of  the  root  cause  of 
software  problems,  Lockheed  Martin  cannot  conduct  effective  corrective  action  to 
address  software  deficiencies. 

Joint  Program  Office  Oversight  of  Lockheed  Martin 

The  CSI  findings  documented  at  Lockheed  Martin,  Northrop  Grumman,  and  BAE  Systems 
indicate  that  JPO  had  not  established  a  CSI  program  for  the  F-35  Program. 

JPO  had  not  flowed  down  a  contractual  requirement  to  Lockheed  Martin  (and 
subsequently,  its  subtier  suppliers]  to  implement  a  CSI  management  program  in 
accordance  with  Public  Law  and  DoD  Policy.  CSIs  require  special  handling,  engineering, 
manufacturing,  and  inspection  documentation  to  control  and  ensure  safety  of  flight. 
Without  a  CSI  program,  there  is  an  increased  safety  risk  to  the  aircraft  and  warfighter. 

Another  major  finding  noted  that  JPO  had  not  funded  Lockheed  Martin  to  create  a 
DMSMS  process  and  had  not  developed  a  DMSMS  Program  Management  Plan;  causing  a 
significant  cost  risk  to  the  program.  However,  according  JPO,  JPO  was  in  the  process  of 
funding  a  DMSMS  program. 

We  also  identified  that  JPO  did  not  establish  definitive  exit  criteria  for  the  block 
2A  Air  System  Test  Readiness  Review,  reducing  the  probability  that  the  system  is  ready 
to  proceed  into  formal  test.  JPO  systems  engineering  and  risk  management  plans 
require  contractors  to  continuously  assess  the  F-35  Program  risks.  However,  JPO  was  not 
reviewing  and  documenting  program  risks  during  the  Program  Management  Advisory 
Board  meetings  or  the  risk-level  management  reviews.  Without  active  participation 
in  systems  engineering,  risk  management,  and  lifecycle  logistics  planning  for  the 
F-35  Program,  JPO  cannot  ensure  that  all  program  requirements  will  be  met. 

Lockheed  Martin  Assessment  Summary 

Our  assessment  of  Lockheed  Martin's  quality  management  system,  processes  and 
procedures  indicated  a  lack  of  discipline  in  complying  with  AS9100  requirements.  This 
will  result  in  nonconforming  hardware,  less  reliable  aircraft,  and  increased  cost. 
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Manufacturing  planning  and  process  qualification  were  not  at  the  level  commensurate 
with  current  and  planned  production  levels.  In  general,  documentation  contained 
inaccurate  and  incorrect  verification  steps  and  lacked  sufficient  detail  to  provide 
adequate  work  instructions.  This  indicates  that  processes  remain  immature  and  that 
assembly  instructions  require  immediate  updating. 

Lockheed  Martin's  management  of  requirements;  including  those  flowed  down  to 
suppliers  in  the  areas  of  design,  configuration  management,  software,  and  FAI;  were  not 
always  clearly  defined,  approved,  maintained,  and  verified  for  compliance.  In  addition,  the 
requirements  provided  conflicting  direction.  Without  clear  and  accurate  requirements, 
Lockheed  Martin  cannot  ensure  that  flight  hardware  meets  F-35  Program  requirements. 

Finally,  Lockheed  Martin  was  not  committed  to  FOD  control.  Workers  were  in  the 
assembly  areas  with  expired  FOD  certification,  and  discipline  was  generally  lacking  in 
FOD-control  and  FOD-critical  areas.  The  inspection  team  noted  substantial  FOD  in 
and  around  the  aircraft;  these  FOD  deficiencies  continued  even  after  Lockheed  Martin 
management  shut  down  the  line  twice  to  correct  noted  deficiencies  during  the  OIG 
assessment.  An  ineffective  FOD  program  can  result  in  damage  to  F-35  aircraft  and  is  a 
safety  flight  issue. 

Northrop  Grumman 

(El  Segundo  and  Palmdale,  California] 

Northrop  Grumman  is  one 
of  Lockheed  Martin's  princi¬ 
pal  subcontractors.  Northrop 
Grumman  manufactures  the 
center  fuselage  and  weapons 
bay  doors  of  the  aircraft. 

The  manufacturing  opera¬ 
tions  are  divided  between 
two  manufacturing  facilities: 

El  Segundo  and  Palmdale, 

California.  The  engineering 
team  is  located  at  the 
El  Segundo  facility.  That  facility  also  produces  the  composite  skins  and  weapons  bay 
doors.  The  Palmdale  facility  develops  and  integrates  the  center  fuselage  component; 
including  the  tail  cap  antenna,  composite  panels  and  covers,  in-flight  operable  doors, 
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arresting  gear  system,  weapons  bay  door  drive  system,  fire  protection  system,  and 
multiple  mission  systems  provided  by  its  suppliers.  Figure  5  shows  the  center  fuselage 
on  the  integrated  assembly  line  at  the  Palmdale  facility. 

Our  assessment  of  Northrop  Grumman  resulted  in  66  findings  that  identified  deficiencies 
in  Northrop  Grumman's  implementation  of  the  AS9100  Quality  Management  System. 
As  part  of  our  assessment  of  Northrop  Grumman,  we  wrote  several  findings  regarding 
the  effectiveness  of  Lockheed  Martin  requirements  flow  down  to  suppliers.  Figure  6 
provides  the  Northrop  Grumman  findings  by  AS9100  clause  with  the  following  sections 
summarizing  significant  issues  documented  during  the  assessment. 

Figure  6.  Northrop  Grumman  Findings 


No.  of  Findings 
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Planning  of  Product  Realization  (7.1) 

Our  assessment  documented  several  deficiencies  in  the  area  of  Product  Planning  and 
Realization.  We  noted  that  the  temperature  and  relative  humidity  parameters  in  the 
automated  paint  system  were  not  programmed  to  required  specifications,  and  operators 
changed  the  parameter  limits  without  management  approval.  In  two  other  instances,  the 
required  torque  value  was  not  included  in  the  process  work  instructions.  Specifically, 
the  torque  paint  application  and  special  hole  finish  verification  were  not  included  in  the 
process  work  instructions.  A  third  finding  documented  a  similar  deficiency  with  adhesive 
application  and  cure  times.  We  could  not  determine  if  these  required  parameters  had 
been  met  or  verified.  Product  reliability  and  repeatability  relies  on  explicit  process  work 
instructions,  and  Northrop  Grumman  needs  to  assess  the  risk  to  and  impact  on  delivered 
hardware  resulting  from  these  deficiencies. 

Customer-Related  Processes  (7.2) 

Customer  Communication  (7.2.3).  Northrop  Grumman  did  not  provide  DCMA  with 
adequate  access  to  Northrop  Grumman  command  media  or  other  enterprise  systems 
to  effectively  administer  its  delegated  contract  oversight  functions.  Specifically,  DCMA 
Palmdale  requires  access  to  corrective  and  preventative  action  databases,  internal  audits, 
metrics,  policies,  processes,  and  procedures. 

The  lack  of  access  to  program  information  prevented  DCMA  from  effectively 
administering  Northrop  Grumman  Palmdale  contracts  as  required  by  the  Federal 
Acquisition  Regulation/Defense  Federal  Acquisition  Regulation  Supplement  and 
monitoring  JPO's  expectations  for  cost,  schedule,  and  performance. 

Design  and  Development  (7.3) 

Design  and  Development  Planning  (7.3.1).  We  documented  deficiencies  with  the 
software  development  processes  for  engineering  software  releases.  Northrop  Grumman 
released  five  engineering  software  versions  of  the  mission  systems  software  to  Lockheed 
Martin  without  a  formal  documented  product  test,  including  the  test  plan  for  changed 
requirements  and  the  test  results.  Instead,  Northrop  Grumman's  practice  was  to  test 
software  releases  only  at  the  unit/module  level.  Northrop  Grumman  stated  that  software 
test  performed  at  the  unit  level  versus  assembly  level  are  done  informally;  therefore, 
no  records  were  retained.  In  addition,  the  Software  Safety  Engineer  was  not  actively 
participating  in  software  product  evaluations  to  review  and  assess  safety  assurance-level 
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requirements  as  required  by  the  Software  Development  Plan.  The  assessment  reviewed 
four  software  product  evaluations;  however,  records  indicated  that  the  Software  Safety 
Engineer  had  participated  in  only  one  of  four  evaluations. 

Software  testing  at  the  unit/module  level  creates  an  integration  risk  because  modular 
testing  may  not  have  been  sufficient  to  verify  software  interoperability  issues. 

Purchasing  (7.4) 

We  cited  deficiencies  in  Northrop  Grumman's  flow  down  of  supplier  requirements  and 
supplier  submittal  of  contract  deliverable  data.  Some  purchase  orders  showed  incorrect 
part  or  material  revision  information.  In  one  example,  the  purchase  order  for  carbon  fiber 
material  did  not  include  specification  revision  information.  Northrop  Grumman  relied  on 
the  composite  material  supplier  to  determine  the  correct  version  of  the  ordered  products. 
Receiving  inspection  personnel  could  not  verify  the  correct  versions  of  parts  or  materials 
received  due  to  the  lack  of  part  revision  information  on  the  purchase  order.  Therefore,  the 
material  was  accepted  and  used  in  production  without  engineering  approval  or  verification 
of  correct  version.  Without  revision  information  in  purchase  orders,  receiving  inspection 
personnel  cannot  verify  that  correct  versions  of  parts  or  materials  are  received,  and  that 
incorrect  parts  and  materials  are  not  introduced  into  production. 

We  sampled  several  of  Northrop  Grumman's  automated  purchase  orders  and  found  that 
the  purchase  orders  did  not  include  the  means  to  identify  specification  revisions  on  raw 
material  receiving  reports.  Because  of  the  significance  of  this  finding,  Northrop  Grumman 
initiated  an  in-depth  investigation  to  determine  the  impact  to  delivered  hardware.  Other 
findings  documented  that  Northrop  Grumman  was  not  following  its  supplier  management 
procedures.  Suppliers  rated  with  a  "red,"  or  high-risk  supplier  scorecard  rating  were  still 
allowed  to  conduct  self-inspections  of  material  shipped  to  Northrop  Grumman.  Northrop 
Grumman  should  have  imposed  additional  inspections  to  ensure  material  conformity. 

Production  and  Service  Provision  (7.5) 

Control  of  Production  and  Service  Provision  (7.5.1).  Our  assessment  found  evidence 
of  incomplete  manufacturing  and  quality  assurance  records  and  a  lack  of  adherence  to 
released  process  procedures.  A  major  finding  noted  that  Northrop  Grumman  did  not 
ensure  that  all  quality  assurance  verification  operations  were  completed.  Northrop 
Grumman  did  not  complete  several  quality  inspection/verification  steps  in  shop  orders 
for  manufacturing  the  Integrated  Power  Package  Door  Mechanism  and  the  Collector  Panel. 
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We  noted  that  Northrop  Grumman  personnel  did  not  follow  released  work  instructions. 
The  operators  were  not  following  the  process  work  instruction  for  setting  the  composite 
cutting  blade  depth  prior  to  the  cut.  The  operator  set  a  depth  that  did  not  agree  with  the 
work  instruction.  In  another  example,  during  center  fuselage  buildup  operations,  fuel 
sealant  had  been  applied  to  incorrect  areas.  The  operator  stated  that  the  area  sealed 
was  no  longer  a  "stay  out  zone"  on  the  hardware;  however,  the  work  instructions  had 
not  been  updated  to  reflect  engineering's  agreement  to  the  change.  In  addition,  two 
records  documenting  that  the  sealant  application  on  several  CTOL  tail  fins  had  not  been 
completed.  Our  assessment  identified  several  more  manufacturing  deficiencies. 

•  Composite  cumulative  adhesive  shelf-life  times  were  not  recorded  for  some 
composite  materials. 

•  The  liquid  shim  application  process  was  not  in  compliance  with 
documented  procedures. 

•  Drill  bit  life  was  not  being  tracked.  Drill  bit  life  should  be  tracked  to  ensure 
dimensional  compliance. 

Production  Process  Verification  (7.5. 1.1).  We  noted  that  Northrop  Grumman  had  not 
performed  or  completed  FAIs  on  all  F-35  center  fuselage  variant  assemblies.  A  review 
of  Northrop  Grumman's  FAI  status  and  schedule  data  noted  that  FAIs  were  initiated 
but  never  completed  for  LRIP  4  variants.  Northrop  Grumman  discontinued  performing 
FAIs  in  accordance  with  Lockheed  Martin's  direction  as  stipulated  in  the  July  20,  2011, 
released  FAI  plan.  The  Lockheed  Martin  FAIs  Plan  is  in  direct  conflict  with  the  SOW. 
Completion  of  FAIs  on  the  F-35  center  fuselage  variant  assemblies  is  essential  to  verify 
that  the  production  process  is  capable  of  producing  assemblies  that  meet  requirements. 

Control  of  Production  Equipment,  Tools,  and  Software  Programs  (7.5. 1.3).  Our 

assessment  noted  deficiencies  with  the  validation  and  control  of  numerical  software 
programs.  More  than  75  percent  of  the  F-35  composite  numerical  control  machining 
programs  were  in  an  "unproven"  state  with  no  traceability  established  between  the 
numerical  control  program  and  the  part  produced.  Specifically,  the  automated  machinery 
screen  used  by  the  operator  to  select  the  file  for  machining  listed  numerous  files,  some 
listed  as  "proven,"  others  as  "unproven."  Numerous  files  designated  as  "unproven"  could 
be  inadvertently  selected  to  generate  the  exact  same  part  number.  A  similar  condition 
existed  for  the  measurement  programs  used  to  validate  the  parts  produced  on  the 
precision  milling  machine.  These  findings  represent  a  significant  risk  that  either 
unproven  files  or  files  not  designated  as  the  latest  approved  configuration  could  be 
selected  and  used  in  producing  the  product. 
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We  documented  tool  control  deficiencies.  The  radio  frequency  identification  system  was 
incapable  of  tracking  tool  location.  It  incorrectly  identified  tools  as  missing  or  in  another 
location;  allowing  similar  tools  to  be  returned  to  incorrect  locations.  In  addition,  Northrop 
Grumman  personnel  did  not  always  trace  missing  tools  as  required  by  procedures. 
Several  other  findings  noted  deficiencies  with  the  identification,  storage,  and  application 
of  tooling.  Ineffective  tool  control  can  lead  to  quality  issues  such  as  the  wrong  tools  used 
during  assembly,  lost  tools,  and  insufficient  visibility  in  calibration  measurement.  It  can 
also  lead  to  field  returns  caused  by  uncalibrated  equipment. 

Preservation  of  Product  (7.5.5).  Our  assessment  identified  deficiencies  with  the  control 
of  limited-life  materials.  Northrop  Grumman's  process  for  controlling  shelf-life  materials 
did  not  ensure  that  all  materials  used  on  the  manufacturing  floor  were  within  their  useful 
life.  Expired  sealant  material  was  used  on  flight  hardware,  and  elastomeric  filler  was 
labeled  with  incorrect  shelf  life  expiration  dates.  Also,  17  tubes  of  expired  product  were 
provided  to  the  production  floor.  We  immediately  notified  Northrop  Grumman,  which 
initiated  action  to  recall  the  nonconforming  materials  and  began  the  material  review 
board  process  for  the  affected  parts.  An  additional  finding  noted  that  material  shelf 
life  information  was  not  always  identified  on  transfer  containers,  leading  to  a  loss  of 
traceability  to  its  useful  life.  Using  expired  materials  could  affect  structural  integrity  of 
the  final  product. 

FOD  procedures  were  not  being  followed  in  production  floor  areas  designated  as 
category  1  FOD-critical  areas.  Specifically,  a  category  1  FOD-critical  area  requires 
heightened  FOD  awareness  including  cordoning  off  the  area,  displaying  FOD-critical 
signs,  and  implementing  sign-in/sign-out  logs  and  tool  control  logs.  In  one  instance, 
crane  operators  in  a  FOD-critical  area  had  not  recorded  themselves  or  their  tooling  into 
the  area.  All  tooling  brought  into  a  FOD-critical  area  are  required  to  be  documented  on  a 
FOD-control  card.  In  another  instance,  personnel  were  performing  a  close-out  operation 
on  an  inverted  fuel  tank  and  side  skins  in  a  category  2  FOD-control  area,  but  this  operation 
should  have  been  performed  in  a  category  1  FOD-critical  area.  The  purpose  of  a  FOD 
program  is  to  ensure  that  loose  hardware  is  accounted  for,  such  as  bolts  or  nuts,  preventing 
the  puncturing  of  tanks,  hydraulic  lines,  or  damaging  other  sensitive  components  during 
aircraft  operation. 
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Control  of  Monitoring  and  Measuring  Equipment  (7.6) 

We  noted  Northrop  Grumman  was  providing  extension  of  calibration  intervals  without 
proper  evaluation  and  engineering  approval.  A  date  extension  was  provided  on  a 
digital  pressure  gauge  with  a  poor  calibration  history  even  though  two  out-of-tolerance 
conditions  were  documented  on  this  item.  Calibrated  equipment  with  a  history  of  out- 
of-tolerance  conditions  should  have  their  calibration  interval  shortened,  not  extended. 

Other  findings  addressed  calibration  deficiencies  such  as  the  use  of  incorrect  calibration 
procedures  and  ambiguous  humidity  requirements  in  the  calibration  lab.  Without  a 
disciplined  and  controlled  calibration  system,  the  accuracy  and  integrity  of  calibrations 
performed  is  uncertain.  This  may  result  in  hardware  being  processed  and  accepted 
because  the  discrepant  test  and  measurement  equipment  masked  an  out-of- 
tolerance  condition. 

Monitoring  and  Measurement  (8.2) 

Monitoring  and  Measurement  of  Product  (8.2.4).  We  found  that  Northrop  Grumman 
did  not  use  the  correct  test  system  configuration  or  software  configuration  for  performing 
electrical  test  methods  on  composite  parts  and  material.  Another  finding  documented 
that  quality  assurance  personnel  did  not  record  or  witness  production  measurements 
and  test  results  on  the  integrated  assembly  line.  We  also  noted  that  work  instructions 
did  not  require  quality  assurance  personnel  to  record  nut  plate  push  test  results  and  rivet 
flushness  measurements;  therefore,  no  test  values  were  recorded  and  test  verification 
could  not  be  done.  Without  proper  test  set-ups  and  quality  assurance  personnel 
verifying  critical  measurements,  Northrop  Grumman  cannot  be  certain  whether  all  the 
requirements  were  tested  and  verified. 

Continuous  Improvement  (8.5) 

Corrective  Action  (8.5.2).  Our  assessment  team  noted  that  the  Northrop  Grumman 
corrective  action  system  for  in-house  operations  and  suppliers  was  ineffective.  Northrop 
Grumman  was  in  the  process  of  transitioning  into  a  new  software  problem  reporting  tool; 
however,  neither  the  legacy  tool  nor  the  new  tool  adequately  captured  the  root  cause 
and  analysis  of  software  nonconformance.  We  also  noted  that  Northrop  Grumman  did 
not  conduct  root  cause  analysis  for  software  internal  audit  findings  and  did  not  approve 
corrective  action  plans  in  a  timely  manner.  Root  cause  analysis  and  timely  corrective 
actions  prevent  future  nonconformances  and  marginal  products  from  being  fielded. 
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Lockheed  Martin  Program  Management  at 
Northrop  Grumman 

Our  assessment  documented  several  major  findings  related  to  Lockheed  Martin's  flow 
down  of  requirements  to  Northrop  Grumman  in  the  areas  of  Software  Traceability, 
Software  Development,  Software  Corrective  Action,  and  FAI. 

Lockheed  Martin  did  not  maintain  mission  systems  requirements  traceability  to 
the  software-level  requirements.  The  Lockheed  Martin  Dynamic  Object  Oriented 
Requirements  System  (DOORS]  requirements  database  contained  about  300  orphan 
requirements  for  missions  systems.  An  orphan  requirement  is  one  that  is  not  derived 
from  or  traceable  to  top-level  system  requirements.  Lockheed  Martin  identified  this 
deficiency  in  July  2010;  however,  it  did  not  document  or  implement  the  root  cause  and 
corrective  action. 

Our  review  of  the  JSF  Program  Air  System  Block  Plan  showed  that  the  plan  had  not  been 
updated  since  August  13,  2008,  and  does  not  reflect  current  block  planning  defining 
Air  Systems  capabilities  for  the  F-35  Program.  As  a  result,  there  is  no  authoritative 
document  defining  current  and  planned  F-35  software  capabilities. 

Northrop  Grumman  Assessment  Summary 

Our  assessment  found  deficiencies  at  both  the  El  Segundo  and  Palmdale  sites.  Deficiencies 
at  the  Northrop  Grumman,  El  Segundo,  site  were  primarily  in  the  area  of  design  and 
purchasing.  Northrop  Grumman's  practice  of  performing  unit/modular  software  testing 
without  documenting  the  results  creates  significant  concerns  regarding  requirement 
verification  and  interoperability.  Without  adequate  design  evaluation  and  test  of  mission 
system  software,  Northrop  Grumman  cannot  ensure  all  software  system  and  safety 
requirements  have  been  met.  Purchase  orders  lacked  material  version  information  to 
ensure  that  material  was  procured  to  the  required  specifications.  After  bringing  this 
deficiency  to  their  attention,  Northrop  Grumman  initiated  an  investigation  to  determine 
the  impact  on  the  hardware. 
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Our  assessment  of  the  Palmdale  facility  noted  several  deficiencies,  most  of  which  were 
attributed  to  the  creation  of  a  new  integrated  assembly  line.  Documentation  of  processes 
and  procedures  used  in  manufacturing  had  not  reached  the  expected  maturity.  Procedures 
did  not  reflect  the  actual  operation,  were  missing  required  detail,  and  required  additional 
proofing  for  accuracy.  Northrop  Grumman  had  implemented  a  radio  frequency  identifier 
system  for  tool  control;  however,  it  was  ineffective  and  did  not  provide  the  necessary 
accountability  and  control  warranted  for  a  complex  production  and  assembly  area. 

BAE  Systems 

(Samlesbury,  United  Kingdom] 

BAE  Systems  is  Lockheed  Martin's 
other  principal  subcontractor. 

BAE  manufactures  the  aft  fuselage 
and  the  carrier  variant  wing-fold 
system;  including  critical  items 
such  as  the  horizontal  tails  and 
vertical  tails.  Additionally,  BAE, 
as  one  of  the  major  software 
suppliers  for  the  F-35  aircraft, 
is  responsible  for  developing  fuel  system  software,  navigation  and  display  software, 
vehicle  system  prognostics  and  health  manager  software,  and  off-board  mission  support 
software.  Figure  7  is  an  image  of  the  aft  fuselage  on  the  manufacturing  floor  at  BAE. 

Our  assessment  of  BAE  resulted  in  82  findings  that  identified  weaknesses  in  BAE's 
implementation  of  the  AS9100  Quality  Management  System.  Additionally,  Lockheed 
Martin  had  several  deficiencies  in  requirements  flow  down  to  BAE.  We  also  identified 
issues  related  to  JPO  and  DCMA  oversight.  Figure  8  provides  BAE  findings  by  AS9100 
clause  with  the  following  sections  summarizing  significant  issues  documented  during 
the  assessment. 
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Figure  8.  BAE  Findings 
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Documentation  Requirements  (4.2) 

Control  of  Documents  (4.2.3).  We  identified  several  production  documents  that  were 
not  under  configuration  control.  Examples  of  nonconfiguration  controlled  procedures 
and/or  documents  available  for  use  include:  chemical  treatment,  paint  identification, 
machining  health  and  safety  plan,  control  of  heat  treatment,  and  bore-cutting  tool 
and  lubricate. 

BAE  updated  the  F-35  quality  assurance  plan  for  LRIP  4  and  5,  but  did  not  submit  the 
documents  to  Lockheed  Martin  for  approval.  Production  process  documentation  not 
approved  or  under  revision  control  included  the  following  checklists:  daily  production 
verification,  first  article  verification,  and  production  process  verification.  Controlling 
such  documentation  creates  the  means  to  ensure  each  assembly  is  exact  and  conforming 
to  requirements. 
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Human  Resources  (6.2) 

Competence,  Training,  and  Awareness  (6.2.2).  BAE  lacked  discipline  in  the  areas  of 
employee  training  requirements,  recertification,  and  training  records.  Skills  matrices 
did  not  always  identify  the  training  required  for  each  operator  to  perform  a  particular 
manufacturing  process.  For  example,  within  building  355,  no  one  had  completed  training 
competency  in  any  of  the  required  technical  skills  for  the  following  critical  processes: 
press  operations,  friction  sawing,  part  marking,  tungsten  inert  gas  welding,  automatic 
grit  blast  machining,  and  radial  arm  drilling.  Many  of  the  personnel  lacking  training 
were  responsible  for  manufacturing  fracture-critical  hardware.  In  addition,  we  found  in 
the  other  manufacturing  facilities  that  BAE  had  not  established  performance  criteria  to 
evaluate  employee  competency.  We  also  found  that  BAE  did  not  have  a  recertification 
program  for  wire  harness  repair. 

A  training  and  certification  program,  to  include  control  of  training  records,  is  critical  to 
ensure  that  personnel  manufacturing  and  inspecting  hardware  are  proficient  to  prevent 
nonconformances.  BAE's  lack  of  a  disciplined  training  program  is  a  significant  concern  to 
the  production  of  F-35  hardware. 

Planning  of  Product  Realization  (7.1) 

Configuration  Management  (7.1.3).  BAE  suppliers  were  not  submitting  minor 
changes  to  BAE  for  review  and  approval  as  required.  For  example,  Honeywell,  a  BAE 
supplier,  approved  an  engineering  change  request  for  the  F-35  Life  Support  System 
without  Lockheed  Martin  or  BAE  approval.  Another  BAE  supplier,  Ametek,  approved  an 
engineering  change  order  for  coating  painting  processes  without  BAE  approval. 

The  assessment  also  noted  that  BAE  suppliers  were  not  submitting  configuration 
status  accounting  records  or  reports  that  provide  the  "as-built"  versus  "as-designed” 
configuration.  For  example,  Honeywell  did  not  provide  a  configuration  status  accounting 
report  for  products  delivered  during  LRIP  4  and  LRIP  5.  Additionally,  Goodrich 
Actuation  also  had  not  provided  a  configuration  status  accounting  report  for  LRIP  5 
delivered  products. 

BAE  did  not  have  schematics/drawings  to  define  the  Fuel  Management  System 
software  test  stand  and  did  not  have  the  set-up/take-down  procedures  or 
troubleshooting  guidelines  necessary  to  maintain  the  test  stand  configuration.  BAE 
also  did  not  conduct  software  certification  of  the  software  test  stand,  as  required 
prior  to  software  qualification  testing. 
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Controlling  engineering  changes  and  providing  adequate  configuration  status  accounting 
of  product  is  essential  to  ensure  the  approved  configuration  replicated  correctly  every 
time.  Failure  to  maintain  the  required  configuration  management  disciplines  increases 
program  performance,  cost,  and  schedule  risks. 

Design  and  Development  (7.3) 

We  identified  several  major  deficiencies  in  the  areas  of  program  management, 
configuration  control,  and  testing.  In  the  area  of  software  development,  BAE  was 
working  to  an  outdated  version  of  the  F-35  mission  systems  software  development 
plan  (SDP),7  was  not  maintaining  the  software  design  document  (SDD)  for  the  vehicle 
system  fuel  management  system,  and  was  working  to  unapproved  software  regression 
test  procedures.  In  the  area  of  hardware  development,  BAE  was  not  approving  all  design 
review  plans  for  the  F-35  Program. 

Design  and  Development  Planning  (7.3.1).  BAE  developed  F-35  software 
without  using  the  latest  Lockheed  Martin  approved  SDP.  According  to  the  contract, 
BAE  is  required  to  follow  the  latest  software  design  plan  released  by  Lockheed  Martin. 
BAE  was  developing  software  to  the  Mission  Systems  Software  Development  Plan 
revision  C  (dated  September  15,  2007),  although  the  current  approved  and  released 
revision  is  revision  E  (dated  August  14,  2012).  Inadequate  flow  down  of  software 
development  requirements  could  affect  aircraft  performance  and  safety  requirements. 

Design  and  Development  Outputs  (7.3.3).  BAE's  software  requirements  specification 
(SRS)  for  the  vehicle  system  fuel  management  system  was  revised  15  times  over  a  5-year 
software  development  period;  however,  the  SDD  was  not  updated  with  these  changes. 
For  example,  the  SDD  was  not  updated  to  reflect  corrective  actions  for  fuel  management 
system  software  trouble  reports  that  were  in  the  revised  SRS.  Software  design  decisions 
and  requirements  changes  should  be  captured  in  the  SDD;  otherwise,  the  software  design 
baseline  may  not  meet  system  performance  or  safety  requirements. 

Design  and  Development  Review  (7.3.4).  We  noted  that  the  life  cycle  management 
design  review  plan  had  not  been  approved  by  the  project  engineering  manager  and 
the  design  review  chairman.  Approval  of  the  plan  certifies  that  the  objectives  and  exit 
criteria  fulfill  the  objectives  of  the  review.  If  the  objectives  and  exit  criteria  in  the  design 
review  plans  are  not  certified,  there  is  no  assurance  that  performance  requirements  will 
be  realized. 


7  The  SDP  describes  a  developer's  plans  for  conducting  a  software  development  effort.  The  term  "software  development"  is 
meant  to  include  new  development,  modification,  reuse,  reengineering,  maintenance,  and  all  other  activities  resulting  in 
software  products.  (DAU) 
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Design  and  Development  Validation  (7.3.6).  We  documented  that  BAE  was  using 
an  unapproved  regression  test  procedure  and  conducted  incomplete  regression  tests. 
Regression  testing  is  important  because  it  provides  confidence  that  changes  do  not 
degrade  functionality.  One  finding  documented  that  there  was  no  approved  regression 
test  procedure  for  the  Prognostics  and  Health  Management  software  release.  Additionally, 
the  functional  test  description  for  the  Prognostics  and  Health  Management  software 
requires  35  test  runs,  but  the  test  report  showed  that  BAE  partially  conducted  only  9  of 
the  35  required  tests.  In  addition,  in  the  software  requirements  verification  matrix, 
four  requirements  were  determined  "unverifiable  by  test"  and  required  verification  by 
analysis;  however,  the  analysis  was  not  performed. 

Purchasing  (7.4) 

BAE's  flow  down  of  requirements  to  suppliers  was  insufficient  to  ensure  that  the 
delivered  product  meets  all  the  necessary  quality  assurance  and  technical  requirements. 
The  assessment  documented  several  findings  citing  inadequacies  in  BAE  flow  down  of 
supplier  requirements  and  suppliers  submittal  of  contract  deliverable  data. 

Purchasing  Information  (7.4.2).  BAE  issued  several  purchase  orders  to  suppliers  that 
contained  inaccurate  and  inconsistent  requirements.  For  example,  two  purchase  orders 
cited  the  wrong  SOW,  and  three  purchase  orders  did  not  specify  Government-furnished 
equipment  by  part  number. 

BAE  also  flowed  down  incorrect  configuration  management  requirements  to  suppliers. 
In  accordance  with  Lockheed  Martin  direction,  the  "Configuration  Management 
Requirements  for  F-35  Suppliers  and  Subcontractors,"  should  have  been  flowed  to 
suppliers  in  purchase  orders  supporting  LRIP  and  follow-on  contracts.  However,  an 
earlier  requirements  document,  titled,  "F-35  Joint  Strike  Fighter  Program,  Configuration 
Management  Plan,"  had  been  flowed  to  suppliers,  but  did  not  contain  the  necessary 
supplier  configuration  management  responsibilities  and  requirements.  Without  proper 
flow  down  of  technical  and  quality  assurance  requirements,  BAE  cannot  ensure  that  their 
products  meet  performance,  reliability,  and  other  system  requirements. 

Verification  of  Purchased  Product  (7.4.3).  BAE  was  not  verifying  that  its  suppliers 
were  submitting  critical  supplier  data  deliverables  such  as  acceptance  test  procedures 
(ATPs),  quality  assurance  plans,  and  configuration  management  certifications. 
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Production  and  Service  Provision  (7.5) 

Control  of  Production  and  Service  Provision  (7.5.1).  We  documented  that  BAE  was 
not  following  process  inspection  procedures,  completing  manufacturing  and  quality 
assurance  records,  or  complying  with  tool  control  processes.  BAE  standard  procedures 
require  that  inspectors  verify  and  record  that  operations  are  complete.  However, 
in  several  instances,  the  inspector  did  not  record  completion  of  the  operations.  For 
example,  we  documented  that  quality  assurance  personnel  had  not  verified  epoxy  primer, 
urethane  topcoat,  and  abrasion-resistant  coating  processes.  In  another  case,  there  was 
no  indication  that  inspectors  had  completed  buy-off  of  the  water  break  test  (ensures  that 
the  surface  is  properly  prepared  prior  to  bonding)  on  the  tail-hook  door  and  aft  section. 

Several  procedures  did  not  contain  adequate  detail  or  requirements  to  control  the 
process.  For  example,  specification  requirements  for  the  application  and  mixing  times 
for  the  epoxy  primer  and  topcoat  were  incorrectly  identified  on  the  paint  shop  process 
documents.  As  a  result,  actual  mix  times  recorded  in  manufacturing  process  documents 
did  not  meet  the  specified  requirements. 

We  found  evidence  of  incomplete  manufacturing  and  quality  assurance  records. 
In  15  instances,  cleaning  and  cure  details  were  not  recorded  in  the  mix  log  books.  We  found 
records  with  insufficient  details  for  paint  pot  life,  mix  time,  and  touch-up  processes.  Mix 
times  and  pot  life  details  are  essential  to  ensure  that  material  properties  meet  specification 
requirements  prior  to  application.  The  failure  to  document  manufacturing  and  inspection 
operations  can  lead  to  incomplete  work,  loss  of  traceability,  and  nonconforming  products. 

Production  Process  Verification  (7.5. 1.1).  A  major  finding  noted  that  BAE  had  not 

performed  or  completed  FAIs  on  all  F-35  details,  subassemblies,  and  assemblies  as 
required  by  Lockheed  Martin  supplier  requirements  and  BAE  internal  instructions.  For 
example,  11  subassembly  FAIs  were  not  conducted  and  three  subassemblies  had  failed 
the  initial  FAI.  A  corresponding  major  finding  was  directed  to  Lockheed  Martin  for 
flowing  down  conflicting  FAI  requirements.  Successful  completion  of  FAIs  on  F-35  details, 
subassemblies  and  assemblies  is  one  of  the  criteria  for  LRIP;  therefore,  BAE's  processes 
do  not  reflect  that  criterion. 

Control  of  Production  Equipment,  Tools,  and  Software  Programs  (7.5. 1.3).  The 

assessment  identified  several  major  deficiencies  documenting  the  inadequate  control 
of  manufacturing  tooling,  jigs,  and  fixtures.  Operators  applied  an  unapproved  steel 
epoxy  stick  in  the  incorrect  location  to  hold  two  parts  on  the  aft  closeout  rib  in  position 
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for  machining  operations  rather  than  using  the  specified  tooling.  We  found  that  BAE 
personnel  were  not  monitoring  tool  wear  of  cutting  tools  in  the  aft  fuselage  assembly 
in  accordance  with  their  production  work  instructions.  To  prevent  quality  defects,  the 
maximum  specified  use  for  a  drill  bit  was  50  drilled  holes;  however,  operators  were  not 
tracking  the  number  of  times  a  drill  bit  was  used. 

We  documented  systemic  issues  with  the  storage,  inventory,  and  accounting  of  tooling. 
Tools  were  often  unaccounted  for  and  not  in  their  correct  locations  as  designated  by  the 
work  cell  tool  inventory  sheets.  A  major  finding  documented  that  BAE  personnel  did 
not  perform  visual,  functional,  and  dimensional  tool  cycle  checks  on  the  final  assembly 
tool  fixture  as  required.  Failure  to  control  manufacturing  tooling  can  lead  to  the  use  of 
discrepant,  damaged,  or  incorrect  tooling. 

Identification  and  Traceability  (7.5.3).  We  documented  that  inventory  lists  located  on 
carts  throughout  the  factory  were  inaccurate.  We  also  found  hardware  commingled,  and 
in  some  cases,  improperly  identified.  As  a  result,  positive  part  identification  and  control 
through  the  assembly  process  was  not  always  evident.  Because  parts  are  manufactured 
for  a  specific  airframe  and  are  not  readily  interchangeable,  failure  to  maintain  traceability 
and  identification  of  those  parts  results  in  interchangeability  issues,  nonconformances, 
and  unnecessary  rework  or  scrap. 

Preservation  of  Product  (7.5.5).  The  assessment  documented  numerous  examples 
of  limited  shelf-life  materials  (LSLM)  improperly  labeled,  tracked,  and  controlled.  For 
example,  BAE  personnel  used  expired  Click  Bond  adhesive  material  in  the  manufacture  of 
vertical  and  horizontal  tail  assemblies.  Other  adhesive  materials  labeled  with  improper 
expiration  dates  were  available  for  use.  In  addition,  a  bottle  of  expired  alodine  material 
was  used  on  the  production  floor  in  the  carbon  subassembly  area.  Shelf-life  material  that 
is  not  properly  labeled  or  controlled  can  lead  to  the  use  of  expired  materials  in  production. 
Use  of  expired  shelf-life  material  compromises  the  assurance  that  its  material  properties 
will  satisfy  the  intended  requirements  and  the  integrity  of  the  assembly. 

Control  of  Monitoring  and  Measuring  Equipment  (7.6) 

BAE  was  not  controlling  calibration  laboratory  processes  to  meet  established 
requirements.  BAE  was  operating  two  calibration  laboratories,  but  neither  laboratory 
was  accredited  to  meet  ISO/IEC  (International  Electrotechnical  Commission)  17025, 
"General  Requirements  for  the  Competence  of  Testing  and  Calibration  Laboratories,"  as 
stipulated  by  BAE's  internal  procedure  and  a  Lockheed  Martin  requirement.  In  addition, 
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several  calibration  procedures  in  the  Test  Instrumentations  Laboratory  did  not  address  all 
requirements  and  criteria  required  by  American  National  Standards  Institute  (ANSI]  Z540, 
"Requirements  for  the  Calibration  and  Measuring  and  Test  Equipment."  For  example, 
the  calibration  procedures  for  temperature  sensors  and  field  test  instrumentation  did 
not  address  measurement  ranges  and  tolerances  for  the  item  to  be  calibrated,  minimum 
performance  requirements,  and  criteria  and  requirements  for  calibration.  Moreover,  BAE 
did  not  have  procedures  to  calculate  the  uncertainty  of  measurements  for  calibrations 
performed  or  a  calibration  procedure  for  furnaces  or  ovens. 

BAE  was  not  following  its  internal  procedures  for  the  tagging  and  removal  of  expired 
monitoring  and  measurement  equipment.  Numerous  examples  of  equipment  with 
calibration  deficiencies  were  found  throughout  the  facility.  Deficiencies  included  out-of- 
calibration  torque  wrenches,  vernier  calipers,  and  a  surface  plate.  Furthermore,  torque 
wrenches  were  used  for  verification  outside  of  their  calibrated  capability.  In  addition,  BAE 
was  not  assessing  the  effect  of  out-of-tolerance  equipment  used  on  production  hardware. 
In  summary,  the  number  of  tool  control  and  calibration  deficiencies  leaves  uncertainty  in 
BAE’s  product  processing  and  acceptance. 

Measurement ,  Analysis ,  and  Improvement  (8.0) 

Control  of  Nonconforming  Product  (8.3)  and  Corrective  Action  (8.5.2).  BAE  is 

closing  material  review  board  documents  without  confirming  effective  and  meaningful 
corrective  action.  Specifically,  four  out  of  ten  material  review  board  dispositions  for 
nonconformances  were  closed  without  implementing  the  specified  corrective  action.  In 
another  finding,  BAE  did  not  submit  all  material  review  dispositions  to  the  onsite  DCMA 
representative  for  concurrence.  Specifically,  a  waiver/deviation  for  the  vertical  tail 
substructure  included  both  nonstandard  repairs  and  use  as  is  dispositions  without  any 
evidence  of  submittal  to  the  DCMA  representative  for  required  approval. 

The  supplier  corrective  actions  process  did  not  always  ensure  suppliers  responded  to 
corrective  action  requests  within  the  required  time  period.  For  example,  21  of  63  supplier 
corrective  action  request  forms  were  past  due  by  as  much  as  200  days.  Furthermore,  the 
supplier  corrective  action  data  were  not  accurately  maintained.  The  supplier  electronic 
corrective  action  database  identified  several  supplier  corrective  action  requests  as  closed; 
however,  a  sample  review  of  the  documentation  indicated  the  status  as  open.  Effective 
review  and  verification  of  material  review  board  dispositions  and  associated  corrective 
actions  are  necessary  to  correct  and  prevent  nonconformances. 
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Lockheed  Martin  Program  Management  at  BAE 

We  reviewed  the  effectiveness  of  Lockheed  Martin  program  management  and  wrote 
seven  findings.  Lockheed  Martin  did  not  communicate  to  BAE  that  the  latest  software 
development  plan  was  revision  E,  released  on  August  14,  2012;  thus,  BAE  was  developing 
JSF  Mission  Systems  Software  to  revision  C,  dated  2007.  The  difference  between  the  C  and 
E  revision  could  have  a  significant  impact  on  functionality  and  performance. 

In  another  finding,  the  F-35  Air  Systems  Software  Development  Plan  requires  that  each 
product-level  software  development  plan  include  plans  for  software  maintenance. 
However,  the  Lockheed  Martin  approved  software  development  plans  for  utilities  and 
subsystems,  prognostics  and  health  management,  ejection  seat  sequencer,  boost/ 
transfer  pump  controller,  and  life  support  system  did  not  include  the  required  software 
maintenance  section.  As  a  result,  there  is  no  assurance  that  software  released  is 
maintainable  after  delivery. 

Lockheed  Martin's  FAI  plan  conflicts  with  the  AS9102  requirements  flowed  down  to  BAE 
in  the  SOW,  resulting  in  BAE  not  performing  FAIs  on  major  assemblies  and  subassemblies. 
Failing  to  perform  FAIs  in  accordance  with  AS9102  could  result  in  the  inability  to  achieve 
process  stability  and  successfully  meet  program  production  rate  goals.  The  lack  of 
process  stability  could  also  result  in  increased  costs  resulting  from  discrepant  hardware 
and/or  the  need  to  perform  additional  product  conformance  verification  to  ensure 
engineering  requirements. 

Lockheed  Martin  did  not  flow  down  the  correct  quality  management  system  requirement 
and  the  correct  nonconforming  material  retention  requirements.  Additionally,  Lockheed 
Martin  did  not  specify  the  delivery  requirements  for  the  Counterfeit  Parts  Prevention 
and  Control  Plan;  therefore,  BAE  did  not  produce  a  plan  as  required  by  contract.  The 
Lockheed  Martin  configuration  management  plan  imposed  on  BAE  identifies  additional 
requirements  for  BAE  to  follow;  however,  several  of  those  references  and  the  processes 
were  no  longer  valid.  Flow  down  of  current  and  correct  requirements  to  suppliers  is 
essential  to  ensure  delivered  products  meet  program  requirements. 

Joint  Program  Office  Oversight  at  BAE 

JPO  did  not  flow  requirements  for  a  CSI  program  to  Lockheed  Martin;  therefore,  BAE 
did  not  implement  the  required  processes  to  ensure  safety  of  flight. 
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DCMA  Oversight  at  BAE 

The  DCMA  UK  was  not  reviewing  BAE  classifications  and  dispositions  of  all  minor 
variances,  nor  participating  in  or  reviewing  the  BAE  Material  Review  Board  actions  as 
required  by  its  LOD.  BAE  processes  an  average  of  20  minor  variations  a  day;  however, 
as  agreed  to  with  BAE,  DCMA  reviews  only  about  10  percent  of  these,  or  40  variances  a 
month.  In  addition,  DCMA  UK  had  not  delegated  the  responsibility  for  review  of  minor 
variances  to  DCMA  office  located  at  BAE  suppliers.  Government  oversight  ensures  that 
engineering  evaluations  and  corrective  actions  occur. 

BAE  Systems  Assessment  Summary 

The  software  tests  review  provided  little  confidence  that  software  was  fully  tested.  There 
was  also  inadequate  communication  and  requirements  flow  down  from  Lockheed  Martin 
to  BAE,  leading  to  software  configuration  management  gaps. 

Supplier  management  practices  failed  to  consistently  define  technical  and  quality 
assurance  requirements  in  purchase  orders,  ensure  supplier  corrective  actions  were 
completed  within  process  timeframes,  and  verify  supplier  data  submittals  were  in 
accordance  with  purchase  orders.  BAE  must  ensure  all  technical  and  quality  assurance 
requirements  are  properly  included  in  contracts  and  implemented  by  their  suppliers  to 
meet  cost,  schedule,  and  performance  goals. 

In  addition,  improved  production  discipline  is  required.  The  assessment  showed  that 
additional  process  discipline  is  required  to  ensure  conformity  to  end-item  requirements. 
Personnel  were  relying  on  their  knowledge  of  the  design  and  production  processes, 
rather  than  following  documented  processes.  We  found  14  examples  of  documentation 
that  lacked  essential  detail  to  define,  implement,  and  capture  production  processes. 

Tool  control,  maintenance  of  precision  tools,  and  calibration  needs  attention.  Uncontrolled 
and  unaccounted  for  tools  were  prevalent  throughout  the  production  facility.  Additionally, 
the  calibration  system  had  significant  deficiencies.  A  disciplined  calibration  program  is 
essential  in  determining  the  performance  and  acceptance  of  product. 
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L-3  Display  Systems 

(Alpharetta,  Georgia] 

L-3  Display  Systems  (L-3],  Alpharetta,  Georgia,  a  division  of  L-3  Communications 
Corporation,  is  a  Lockheed  Martin  subcontractor  that  manufactures  the  Panoramic 
Cockpit  Display  (PCD]  for  the  F-35  Lightning  aircraft.  The  PCD,  Figure  9,  performs 
significant  functions  such  as  providing  pilot  flight  navigation  and  sensor  information, 
communication,  and  threat  situational  awareness.  The  PCD  consists  of  two  major 
components,  the  Display  Unit  (DU]  and  the  Electronics  Unit  (EU].  The  DU  is  the  interface 
between  the  pilot  and  the  aircraft  situational  awareness  systems  while  the  EU  is  the 
electronics  package  capturing  and  providing  data  to  the  DU  from  aircraft  systems. 

Figure  9.  L-3  Communications  Panoramic  Cockpit  Display. 

Source:  Image  courtesy  of  L-3  Display  Systems 


Our  assessment  of  L-3  resulted  in  56  findings  that  identified  deficiencies  in  L-3's 
implementation  of  the  AS9100  Quality  Management  System.  Figure  10  provides  L-3 
findings  by  AS9100  clause  with  the  following  sections  summarizing  significant  issues 
documented  during  the  assessment. 
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Figure  10.  L-3  Communications  Findings 
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Documentation  Requirements  (4.2) 

Our  assessment  of  L-3's  document  control  system  identified  several  findings  with  review, 
revision  control,  approval,  and  distribution  of  items  such  as  internal  procedures  and 
manufacturing  instructions.  We  found  some  documentation  did  not  have  management 
approval,  were  released  without  periodic  review,  had  inaccurate  revisions  and  dates,  or 
document  indexes  that  were  not  maintained.  Check  sheets  used  to  define  the  buildup 
of  circuit  cards  in  the  surface-mount  technology  assembly  area  were  not  controlled 
and  there  was  no  means  to  verify  the  latest  revision.  This  lack  of  accurate,  complete, 
and  current  documentation  could  adversely  affect  the  ability  of  the  F-35  Program  to 
deliver  products  that  meet  customer  performance  requirements  in  accordance  with 
schedule  requirements. 
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Configuration  Management  (7.1.3) 

Our  assessment  documented  several  findings  pertaining  to  configuration  management. 
We  identified  a  lack  of  baseline  technical  documentation  for  L-3  DU  and  EU  products. 
According  to  the  L-3  product  configuration  documentation,  a  documented  product 
baseline  was  required  concurrently  with  the  first  production  aircraft  delivered  in 
May  2011;  however,  this  was  not  done.  Similarly,  an  allocated  baseline  configuration  was 
required  for  CDR  closure,  but  it  also  was  not  done.  A  configuration  baseline  establishes  a 
product's  form,  fit,  and  functional  performance  that  is  analyzed,  tested,  and  characterized. 
Such  a  rigorous  approach  was  lacking,  and  could  result  in  additional  product  cost. 

L-3's  engineering  change  control  process  lacked  discipline.  L-3  incorrectly  classified 
engineering  changes  that  affect  the  form,  fit,  and  function  as  "minor"  changes  when  they 
should  have  been  classified  as  "major."  Major  engineering  changes  require  Lockheed 
Martin  approval.  For  example,  L-3  classified  two  changes  as  "minor"  that  increased  the 
weight  of  the  display  interface  circuit  card  assembly  and  weight  of  the  touch  screen  by 
85  percent  and  62  percent,  respectively.  These  changes  affected  product  form,  fit,  or 
function  and  should  be  classified  as  "major"  and  not  as  "minor."  We  identified  that  L-3 
implemented  several  "major"  changes  prior  to  receiving  Lockheed  Martin  authorization. 
Incorrect  classification  of  engineering  changes  and  the  implementation  of  engineering 
changes  prior  to  authorization  bypass  the  systems  engineering  review  and  approval 
cycle.  Inadequate  reviewofengineering  changes  will  adversely  affect  product  performance, 
quality,  and  reliability. 

Design  and  Development  (7.3) 

We  found  that  the  electromagnetic  interference  (EMI]  qualification  for  the  PCD,  EU,  and 
DU  did  not  meet  the  performance-based  specification,  frequency  bandwidth  requirement. 
The  performance-based  specification  requires  that  radiated  susceptibility  testing 
be  performed  over  a  specific  range  of  frequencies  (10  KHz  to  18  GHz]  in  accordance 
with  MIL-STD-461E,  "Requirements  for  the  Control  of  Electromagnetic  Interference 
Characteristics  of  Subsystems  and  Equipment."  However,  L-3  was  performing  testing  in 
the  range  of  30  MHz  to  18  GHz  without  engineering  approval  for  the  reduced  frequency 
range.  Although  the  EMI  test  report  was  comprehensive,  the  quality  assurance  review 
process  failed  to  identify  that  a  performance-based  requirement  was  not  being  met. 
Additionally,  according  to  the  documentation,  the  EMI  qualification  test  was  completed 
without  issues. 
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L-3  did  not  adequately  review  the  F-35  PCD  integrated  system  verification  plan/ 
procedures.  The  PCD  Functional  Test  Procedure,  Software  Functional  Test  Description, 
and  Requirements  Verification  Matrix  had  not  been  reviewed  or  dispositioned.  We 
also  found  that  Lockheed  Martin,  as  the  prime  contractor,  should  have  approved  these 
documents  but  did  not. 

Product  and  Service  Provisions  (7.5) 

Control  of  Production  and  Service  (7.5.1).  Quality  assurance  personnel  did  not  verify 
several  DU  ATP  luminance8  requirements  to  the  updated  requirement.  Specifically,  black 
nonuniformity  and  red,  green,  blue,  and  white  luminance  nonuniformity  were  required 
not  to  exceed  the  value  of  30  percent.  L-3  indicated  to  Lockheed  Martin  that  it  could 
not  reasonably  meet  the  specification,  so  Lockheed  Martin  issued  a  variance  allowing  the 
value  not  to  exceed  50  percent.  Although  Lockheed  Martin  granted  requirement  relief, 
the  L-3  test  procedure  did  not  reflect  the  change,  and  quality  assurance  personnel  did  not 
verify  the  new  requirement. 

Production  Process  Verification  (7. 5. 1.1).  Our  findings  identified  deficiencies  with 
L-3's  supplier  FAI  program.  L-3  did  not  require  all  selected  suppliers  to  perform  FAIs. 
Specifically,  the  JSF  contract  provision  for  the  LRIP  5  states  that  FAI  requirements  shall 
be  flowed  down  to  L-3  suppliers.  Two  purchase  orders  for  suppliers,  Ayesas  and  DDI 
Sales  Corporation,  required  FAI  on  their  product.  However,  the  supplier  did  not  perform 
FAIs,  and  L-3  accepted  the  hardware  without  suppliers  demonstrating  the  capability  to 
produce  conforming  products.  As  an  example,  we  found  a  supplier  FAI  report  containing 
several  nonconformances  that  were  not  addressed,  although  L-3  had  been  accepting 
hardware  from  this  supplier  since  2008.  Finally,  L-3  was  not  maintaining  all  supplier 
FAI  quality  assurance  records  in  accordance  with  contractual  requirements.  L-3's  lack 
of  systems  engineering,  regarding  FAIs,  creates  uncertainty  that  components  built  in  the 
future  will  conform  to  requirements. 

Control  of  Production  Equipment,  Tools,  and  Software  Programs  (7.5. 1.3). 

L-3  did  not  maintain  EU  and  DU  test  stations  to  drawing  requirements.  Of  the 
five  test  stations  in  operation,  one  test  station  did  not  have  a  released  drawing  package 
defining  the  equipment,  equipment  wiring,  software,  and  verification  of  accuracy.  Another 
test  station  had  a  released  drawing  but  the  test  station  was  not  configured  in  accordance 
to  the  released  drawing.  In  addition,  we  could  not  find  evidence  that  similar  test  stations 


Merriam-Webster  dictionary  defines  luminance  as:  "the  luminous  intensity  of  a  surface  in  a  given  direction  per  unit  of 
projected  area." 
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would  produce  comparable  results.  We  found  the  method  for  performing  correlation 
analysis  among  the  test  stations  was  undefined.  Correlation  studies  should  be  performed 
to  quantify  the  variability  between  multiple  test  stations  used  for  final  acceptance  test. 

Preservation  of  Product  (7.5.5).  Our  assessment  team  identified  four  findings 
regarding  the  control  of  LSLM,  FOD,  and  electrostatic  discharge  (ESD)  processes  and 
practices.  We  found  LSLM  items  on  the  manufacturing  floor  not  stored  at  the  required 
temperature,  improperly  identified,  or  expired.  Additionally,  L-3  did  not  fully  implement 
or  follow  its  product  protection  program  and  requirements  for  FOD  and  ESD.  DUs  were 
stored  on  transport  carts  without  adequate  FOD  and  ESD  protection.  In  addition,  L-3 
did  not  perform  verification  of  the  conductive  flooring  and  constant  wrist  strap  monitors 
in  accordance  with  its  ESD  Compliance  Verification  Plan.  This  lack  of  preservation  of 
product  could  result  in  reduced  product  quality  and  reliability. 

7.6  Monitoring  and  Measurement  Equipment 

L-3  could  not  demonstrate  adherence  to  calibration  standards.  The  current  Lockheed 
Martin  purchase  order  requires  L-3  to  have  a  calibration  program  that  meets  one  of  the 
three  industry  standard  requirements:  ISO  10012-1,  "Requirements  for  Measurement 
Processes  and  Measuring  Equipment,"  ISO  17025,  "General  Requirements  for  the 
Competence  of  Testing  and  Calibration  Laboratories,”  or  ANSI  Z540-3,  "Requirements  for 
the  Calibration  of  Measuring  and  Test  Equipment." 

We  found  seven  examples  of  out-of-calibration  equipment  still  in  use  on  the  production 
floor  or  available  for  use.  Four  pieces  of  equipment  were  in  the  calibration  area  and 
not  properly  identified  and  segregated.  Three  others  were  available  for  use  on  the 
production  floor  with  only  one  of  the  three  tagged  as  out-of-calibration.  L-3  could  not 
provide  objective  evidence  that  an  out-of-calibration  piece  of  equipment  was  traceable 
to  the  flight  units  tested,  as  evidenced  by  evaluating  three  flight  unit  acceptance  records. 
L-3's  calibration  and  control  system  failed  to  provide  adequate  corrective  actions.  As  an 
example,  engineering  personnel  evaluated  an  out-of-tolerance  photometer  requiring  the 
next  four  photometers  to  be  evaluated  to  the  internal  lighting  standard.  We  found  no 
documentation  or  evidence  that  the  evaluation  had  occurred. 
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L-3's  dimensional  final  inspection  methods  and  measurement  equipment  were 
incapable  of  verifying  product  conformance  to  drawing  requirements.  The  ATP  required 
the  inspector  to  use  "single-dimension  vernier  calipers"  to  measure  a  multi-dimension 
geometric  parameter  on  the  DU.  However,  this  measurement  equipment  and  method  was 
incapable  of  measuring  the  required  dimensions. 

Our  findings  indicate  that  the  systemic  issues  with  the  control  of  monitoring  and 
measurement  equipment  could  significantly  affect  measurement  accuracy  and/or  quality 
assurance  data  used  to  determine  product  acceptance. 

Lockheed  Martin  Program  Management  at  L-3 

We  reviewed  the  effectiveness  of  Lockheed  Martin  program  management  because  it  is 
the  prime  contractor  ultimately  responsible  to  integrate  L-3  products.  We  found  that 
the  performance-based  specification  for  the  aircraft  PCD  flowed  down  from  Lockheed 
Martin  to  L-3  had  incomplete,  conflicting,  and  ambiguous  system  requirements.  There 
were  action  items  from  CDR  pertaining  to  the  requirements  that  were  not  formally 
closed,  leading  to  the  document  ambiguity.  The  Lockheed  Martin  corrective  action 
system  implemented  at  L-3  failed  to  effectively  eliminate  nonconformities  and  prevent 
their  recurrence.  Lockheed  Martin  accepted  PCDs  without  L-3  having  performed  FAI 
and  did  not  require  L-3  to  perform  configuration  status  accounting  for  components 
delivered  for  LRIP  units  1  through  5,  that  were  previously  delivered.  Lockheed  Martin 
is  not  approving  or  disapproving  all  major  changes  nor  providing  concurrence  on  the 
classification  submitted  by  L-3,  providing  further  evidence  that  Lockheed  Martin  was  not 
sufficiently  managing  its  suppliers. 

DCMA  Oversight  at  L-3  Communications 

We  found  that  DCMA  Fort  Worth,  the  prime  DCMA  office  for  the  program,  and  DCMA 
Atlanta  did  not  execute  all  responsibilities  specified  in  the  memorandum  of  agreement 
for  the  F-35  Program.  DCMA  Fort  Worth  delegated  the  administration  of  Government 
contract  quality  assurance  of  all  F-35  parts  produced  at  L-3  to  DCMA  Atlanta  in  a  letter 
of  delegation  (LOD],  with  the  requirement  that  Atlanta  perform  risk-based  surveillance 
on  high-risk  components.  During  our  review  of  DCMA  Atlanta  records,  we  found  no 
evidence  that  surveillance  was  being  conducted  on  F-35  products  within  the  L-3  facility. 
Furthermore,  DCMA  Atlanta  was  not  providing  the  prescribed  reports  requested  in  the 
LOD  such  as  quality  escapes,  supplier  risks,  corrective  action  requests,  and  monthly 
Product  Assessment  Reports. 
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L-3  Communications  Assessment  Summary 

We  found  deficiencies  in  most  of  the  AS9100  Quality  Management  System  disciplines.  The 
most  significant  deficiencies  highlight  the  lack  of  process  discipline  in  implementation  of 
Product  Realization,  which  includes  the  following  categories:  configuration  management, 
product  verification,  and  monitoring  and  measurement  equipment. 

The  implementation  of  L-3's  configuration  management  system  was  inadequate  and 
could  lead  to  inconsistencies  in  product  performance,  functionality,  and  reliability.  The 
L-3  DU  and  EU  development  program  did  not  produce  product  configuration  baselines 
prior  to  delivery  of  the  first  production  unit.  L-3  had  incorrectly  classified  engineering 
changes  that  affected  form,  fit,  and  function,  such  as  increases  to  weight  and  human 
interface  constraints,  and  had  not  received  Lockheed  Martin  approval  for  major  design 
changes  prior  to  manufacturing  the  product. 

L-3's  lack  of  process  discipline  in  the  area  of  product  verification  was  particularly 
disconcerting.  We  noted  systematic  breakdowns  in  requirements  verification,  FAI, 
test  station  configuration,  and  preservation  of  product  that  raises  questions  about  the 
integrity  of  the  L-3  product.  L-3  did  not  consistently  verify  that  product  acceptance  test 
data  met  specified  requirements  and  did  not  consistently  review  FAI  reports.  In  addition, 
L-3  did  not  manage  or  maintain  test  station  configurations.  L-3  also  insufficiently 
implemented  LSLM,  FOD,  and  ESD  processes. 

L-3  did  not  demonstrate  adherence  to  calibration  standards.  L-3  did  not  ensure  out- 
of-calibration  equipment  was  identified  and  removed  from  the  production  floor.  L-3's 
calibration  system  failed  to  recall  flight  hardware  after  discrepant  test  equipment 
was  identified. 
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Honeywell  Aerospace 

(Yeovil,  United  Kingdom] 

Honeywell  is  a  subcontractor  to  BAE  and  a  tier  2  subcontractor  to  Lockheed  Martin. 
Honeywell  manufactures  the  F-35  aircraft  life  support  system,  consisting  of  the  On-Board 
Oxygen  Generation  System  [OBOGS],  Backup  Oxygen  System  [BOS],  and  Service 
Connection  Package.  Figure  11  is  a  diagram  of  the  Life  Support  System. 

Figure  11.  Diagram  of  the  Life  Support  System. 

Source:  Image  courtesy  of  Honeywell 


Our  assessment  of  Honeywell  resulted  in  38  findings  that  identified  deficiencies  in 
Honeywell’s  implementation  of  the  AS9100  Quality  Management  System.  Figure  12 
provides  Honeywell  findings  by  AS9100  clause  with  the  following  sections  summarizing 
significant  issues  documented  during  the  assessment. 
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Figure  12.  Honeywell  Findings 

No.  of  Findings 
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Human  Resources  (6.2) 

Competence,  Training,  and  Awareness  (6.2.2).  The  Honeywell  training  program  was 
inadequate.  Honeywell  does  not  have  an  overarching  procedure  to  govern  the  training 
requirements,  training  roles  and  responsibilities,  and  records  retention  requirements. 
The  Human  Resource  Manager  explained  that  although  Honeywell  delegated  the  various 
assembly  floor  training  needs  and  records  to  the  respective  floor  managers;  in  practice, 
each  manager  had  a  different  approach.  As  a  result,  not  all  records  and  certifications  were 
retained  or  readily  accessible.  Additionally,  we  noted  personnel  were  manufacturing 
F-35  hardware  with  expired  soldering  and  ESD  certifications.  Training  ensures  personnel 
proficiency  in  industry  best  practices,  techniques,  and  methods.  Without  a  rigorous 
training  program,  Honeywell  cannot  ensure  the  products  delivered  to  BAE  conform  to 
expected  workmanship  standards. 
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Planning  of  Product  Realization  (7.1) 

Configuration  Management  (7.1.3).  Honeywell  did  not  control  software  configuration 
changes  made  to  the  oxygen  monitor  software.  Honeywell  was  making  changes  to  the 
released  oxygen  monitor  software  using  the  developmental  change  control  processes 
without  the  necessary  reviews,  classifications,  and  verifications  required  for  released 
software.  We  found  that  changes  were  approved  without  identifying  the  actual 
change  implemented,  the  severity  of  change,  the  impact  on  software  requirements  or 
specifications,  and  the  level  of  retesting  required  for  the  change.  Honeywell’s  software 
configuration  control  process  poses  a  risk  to  system  integration  and  performance  because 
the  changes  were  not  evaluated  by  engineering  and  management. 

Design  and  Development  (7.3) 

Design  and  Development  Inputs  (7.3.2).  Honeywell  conducted  Preliminary  Design 
Reviews  (PDRs)  and  CDRs  for  the  life  support  system,  chemical  biological  filtered  air 
system  subassembly,  and  pilot  interface  connector  without  approved  specifications.  We 
noted  that  the  Oxygen  Monitor  system  software  functions  were  not  traceable  to  system 
requirements  and  have  not  been  validated  or  verified.  These  functions  included  self¬ 
diagnostics,  monitoring  and  control,  and  operating  system  interface  standards.  Without 
requirements  definition,  traceability,  and  verification;  Honeywell  cannot  ensure  all  the 
F-35  requirements  for  production  have  been  met. 

Design  and  Development  Outputs  (7.3.3).  Honeywell's  software  Version  Description 
Document  (VDD)9  for  the  Oxygen  Monitor  system  software  does  not  adequately  reflect 
the  software  configuration  changes  implemented  and  lacks  other  pertinent  information 
regarding  the  released  software  build.  We  found  that  the  VDD  did  not  identify  software 
configuration  changes  in  the  software  baseline,  did  not  state  the  new  firmware 
configuration  tested,  and  did  not  reference  the  required  interface  specifications. 
As  a  primary  document  used  for  configuration  identification  of  the  software,  it 
lacked  the  principal  elements  to  provide  BAE  sufficient  insight  into  Honeywell's 
software  configuration. 

Design  and  Development  Review  (7.3.4).  We  documented  that  the  chemical 
and  biological  filtered  air  supply  test  readiness  review  was  not  conducted  prior  to 
qualification  testing  in  accordance  with  Honeywell’s  procedure.  The  purpose  of  a  test 
readiness  review  is  to  provide  management  with  the  assurance  that  the  requirements, 


9  Institute  of  Electrical  and  Electronics  Engineers  STD  610.12  defines  VDD  as,  "The  document  that  accompanies  and 
identifies  a  given  version  of  a  system  or  component.  Typical  contents  include  an  inventory  of  system  or  component 
parts,  identification  of  changes  incorporated  into  this  version,  and  installation  and  operating  information  unique  to  the 
version  described." 
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proposed  method  of  verification,  the  test  methods,  and  equipment  to  be  tested  are 
mature  enough  to  commence  testing.  We  also  found  that  there  were  still  13  of  14 
open  action  items  from  CDR.  Open  action  items  included  updating  the  technical 
procurement  specification  to  include  acceptance  testing  and  cleanliness  level 
requirements,  obtaining  critical  technical  analysis  for  system  safety,  and  determining 
how  to  protect  against  contamination  and  electromagnetic  interference.  Honeywell 
management  elected  to  proceed  into  initial  production  without  the  chief  engineer's 
approval  in  accordance  with  their  design  review  procedure. 

Purchasing  (7.4) 

We  found  circuit  board  assembly  purchase  orders  that  did  not  include  the 
supplemental  requirements  for  electronic  and  electrical  components,  such  as  defining 
lead  and  lead-free  finishes,  electronic  solder  requirements,  and  electronic  markings.  We 
also  found  that  personnel  were  not  verifying  product  technical  data  sheets  as  part  of  the 
product  acceptance  process.  Specifically,  Honeywell  accepted  a  pressure  switch  gage 
using  only  a  certificate  of  conformance  even  though  an  acceptance  test  data  sheet  was 
also  required  for  acceptance.  The  lack  of  supplier  performance  requirement  verification 
could  result  in  quality  escapes. 

Production  and  Service  Provision  (7.5) 

Control  of  Production  and  Service  Provision  (7.5.1).  We  found  that  personnel 
responsible  for  performing  sampling  of  supplier  components  were  not  following  the 
required  sampling  plan.  For  instance,  the  sampling  plan  required  a  minimum  sample 
of  20  items;  but  only  1  item  was  sampled.  In  another  instance,  an  operator  removed  a 
protective  film  prior  to  machining  operations  in  contradiction  of  the  drawing  notes. 

We  found  that  the  procedures  for  handling  of  environmentally  sensitive  material  were 
insufficient.  Honeywell  personnel  were  not:  recording  and  monitoring  sealing  compound 
pot  life,  recording  high-strength  adhesive  film  freezer  out  times,  and  recording  and 
monitoring  cure  times  for  sealing  compounds.  Additionally,  Honeywell  personnel  did  not 
maintain  production  equipment  and  tooling  in  accordance  with  established  procedures: 
soldering  irons  were  not  checked  on  a  monthly  basis,  visual  and  leak  test  record  sheets 
were  not  kept,  equipment  safety  checks  were  not  always  performed,  and  production 
tooling  was  inadequately  identified. 

Preservation  of  Product  (7.5.5).  Honeywell  was  not  ensuring  personnel  were 
following  ESD  and  FOD  procedures  in  the  manufacturing  and  test  areas.  Oxygen  monitor 
controllers  were  not  stored  in  ESD-approved  bags,  static  generators  (e.g.,  a  computer 
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keyboard,  telephone,  and  computer  monitor]  were  in  close  proximity  to  ESD  mats,  and  a 
nonconductive  production  floor  was  used  where  ESD-sensitive  items  were  processed.  In 
the  FOD-control  area,  we  found  that  tools  were  not  placed  back  on  shadow  boards  and 
there  was  no  accountability  of  piece  parts  (small  nuts,  bolts].  Industry  experience  has 
shown  that  strict  adherence  to  ESD  and  FOD  procedures  can  prevent  costly  latent  defects 
and  hardware  nonconformances. 

Control  of  Monitoring  and  Measuring  Equipment  (7.6) 

Honeywell’s  control  of  monitoring  and  measuring  equipment  was  inadequate  as 
evidenced  by  the  following:  uncontrolled  environmental  conditions  in  the  calibration 
laboratory  and  processing  areas;  uncalibrated  tools  in  production,  test,  inspection,  and 
metrology  areas;  and  no  maintenance  of  software  for  measuring  equipment.  We  found 
uncalibrated  pin  gages  in  the  receiving  inspection  area,  a  digital  voltmeter  in  the  test  lab, 
air  pressure  gauges  in  the  OBOGS  assembly  area,  and  a  supply  pressure  regulator  in  the 
calibration  lab. 

Control  of  Nonconforming  Product  (8.3) 

Honeywell  was  not  accurately  completing  quality  notifications  [QNs]  used  to 
document  hardware  nonconformances.  We  found  that  two  nonconformances  were 
dispositioned  as  repaired;  however,  a  repair  had  not  occurred.  Honeywell  also 
dispositioned  another  QN  without  documenting  the  root  cause  and  corrective  action. 
Accurate  QNs  are  necessary  to  prevent  recurrence  of  the  nonconformance. 

Analysis  of  Data  (8.4) 

Honeywell  personnel  implemented  acceptance  test  procedural  changes  without  the 
required  BAE  review  and  approval  for  the  oxygen  concentrator  and  OBOGS.  Additionally, 
several  ATP  waivers/deviations  were  incorrectly  categorized  as  minor  rather  than 
major.  Requirement  changes  defined  in  either  ATPs  or  waivers/deviations  directly  affect 
product  performance  and  should  be  evaluated  by  BAE  engineering. 

BAE  Systems  Program  Management  at  Honeywell 

In  performing  quality  assurance  assessments  of  a  supplier,  we  identified  some  findings 
against  Honeywell's  customer,  BAE.  We  documented  that  BAE's  SOW  to  Honeywell 
for  LRIP  4  and  LRIP  5  were  not  under  configuration  control  and  did  not  contain  the 
appropriate  document  number.  As  a  result,  SOWs  could  not  be  traced  to  the  appropriate 
contract,  creating  a  risk  that  Honeywell  may  be  manufacturing  products  to  obsolete  or 
incorrect  program  requirements. 
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Honeywell  Assessment  Summary 

Our  assessment  indicated  an  immature  product  design  and  production  processes. 
The  design  review  process  allowed  the  OBOGS  program  to  enter  CDR  without  firm 
specifications,  exit  CDR  with  13  open  actions,  and  allowed  production  to  proceed 
with  unapproved  design  documents.  Additionally,  verification  of  hardware  and 
software  performance  requirements  was  not  conducted  to  the  approved  acceptance 
test  requirements. 

Honeywell  failed  to  communicate  effectively  to  BAE  the  impact  of  changes  and 
variances  to  the  design.  In  several  instances,  Honeywell  failed  to  obtain  the  necessary 
review  and  approval  from  BAE  for  changes  to  the  Oxygen  Concentrator  acceptance  test 
procedures  and  Oxygen  Monitor  software  systems.  Additionally,  several  variances  were 
incorrectly  categorized  as  minor  when  they  should  have  been  major,  and  should  have 
been  submitted  to  BAE  for  approval. 

Process  discipline  issues  were  prevalent  throughout  the  facility.  We  found  numerous 
procedural  and  implementation  issues  in  the  areas  of  ESD,  FOD,  calibration,  and  control  of 
LSLM.  The  issues  highlight  quality  assurance  and  process  discipline  concerns  that  could 
introduce  unscheduled  rework  and  repair,  leading  to  cost  overruns  and  schedule  slips. 

United  Technologies  Corporation,  Aerospace  Systems 

(Fort  Worth,  Texas,  and  Independence,  Ohio) 

UTAS  is  a  Lockheed  Martin  subcontractor  that  manufactures  the  landing  gear  for  the  F-35 
Lighting  Aircraft.  The  landing  gear  assembly  line  recently  moved  from  Independence, 
Ohio,  to  Fort  Worth,  Texas.  The  move  began  in  January  2012,  with  the  first  landing  gear 
assembly  delivered  from  the  new  facility  in  October  2012.  In  July  2012,  UTAS  purchased 
the  Goodrich  division  that  was  the  original  manufacturer  of  the  F-35  landing  gear.  At  the 
time  of  our  assessment,  all  the  engineering  offices  were  still  in  Ohio;  only  the  assembly 
line  and  supporting  activities  had  moved  to  Texas.  UTAS  provides  landing  gear  for  all 
three  F-35  variants:  CTOL,  CV,  and  STOVL. 

The  assessment  of  UTAS  resulted  in  51  findings  that  identified  deficiencies  in  UTAS' 
implementation  of  the  AS9100  Quality  Management  System.  The  findings  identified 
issues  with  DCMA  delegations  and  their  implementation  along  with  Lockheed  Martin's 
insufficient  supply  chain  management  of  critical  suppliers.  Figure  13  provides  UTAS 
findings  by  AS9100  clause  with  the  following  sections  summarizing  significant  issues 
documented  during  the  assessment. 
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Figure  13.  UTAS  Findings 
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Documentation  Requirements  (4.2) 

We  found  that  the  quality  assurance  inspection  control  plans  used  in  the  inspection 
of  hardware  were  not  under  configuration  control.  Four  quality  assurance  inspection 
control  plans  found  on  the  production  floor  had  been  updated  by  engineering  but  not 
signed  or  released  in  accordance  with  UTAS  procedures.  Thirty-seven  other  documents, 
such  as  quality  assurance  documents,  landing  gear  documents,  and  engineering  quality 
assurance  procedures;  had  not  been  reviewed  for  adequacy  or  released  in  accordance 
with  their  procedures. 

Human  Resources  (6.2) 

Competence,  Training,  and  Awareness  (6.2.2).  We  found  that  UTAS  used  multiple 
training  systems  that  were  not  maintained,  resulting  in  incomplete  training  requirements, 
incomplete  records,  and  the  employees'  training  effectiveness  not  being  assessed.  We 
found  that  training  records  were  incomplete,  lacking  training  requirements  and  due  dates. 
The  document  system  left  uncertainty  that  employees  had  all  the  necessary  training. 
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Planning  of  Product  Realization  (7.1) 

We  documented  some  tools  and  equipment  that  had  not  been  proof  loaded.  Specifically, 
one  crane  with  various  riggings  was  not  proof  loaded  or  certified  for  the  F-35  landing  gear 
assemblies.  This  did  not  meet  safety  and  occupational  health  requirements. 

Configuration  Management  (7.1.3).  We  found  that  UTAS  did  not  certify  or  update  its 
configuration  management  policy  to  reflect  the  Lockheed  Martin  requirements.  Change 
control,  classification  of  changes,  and  notification  to  the  customer  of  pending  changes 
were  not  accomplished.  For  instance,  Lockheed  Martin's  configuration  management 
requirements  stated  that  a  change  to  solder  composition  should  be  a  major  change; 
however,  this  requirement  was  not  included  in  the  UTAS  configuration  management  plan. 

Customer-Related  Processes  (7.2) 

Customer  Communication  (7.2.3).  UTAS  Fort  Worth  did  not  submit  minor  variances 
to  DCMA  for  review  and  approval.  Lockheed  Martin  requires  that  both  major  and 
minor  variances  be  submitted  to  DCMA  for  evaluation  to  ensure  the  classification  is 
correct  and  is  appropriately  dispositioned.  We  found  that  minor  variances  had  not  been 
submitted;  therefore,  the  classification  of  nonconformances  and  dispositions  may  not  be 
technically  correct. 

Design  and  Development  (7.3) 

We  found  that  UTAS  had  not  completed  the  corrosion  protection  plan  for  the  F-35 
CV  landing  gear  in  accordance  with  the  SOW.  The  corrosion  protection  plan  is  necessary 
to  define  adequate  design  and  maintenance  requirements  for  the  landing  gear  to  ensure 
performance  in  a  high  humidity  and  salt  spray  environment.  We  also  identified  that  UTAS 
had  not  received  Lockheed  Martin's  approval  for  10  item  development  specifications 
and  an  interface  control  document  (ICD)  prior  to  the  start  of  LRIP.  Items  such  as  several 
actuators  and  wire  harnesses,  a  swivel,  switch,  pressure  transducer,  fluid  sensor,  and  the 
electrical  interface  drawing  had  not  been  approved.  The  unapproved  item  development 
specifications  dated  back  to  2004,  and  the  unapproved  electrical  ICD  dated  back  to  2009. 

Additionally,  we  identified  that  the  UTAS  testing  program  did  not  ensure  all  technical 
performance  requirements  were  verified.  For  example,  not  all  Brake/Skid  Control 
System  performance-based  specification  requirements  were  part  of  the  qualification  test 
program.  In  addition,  the  maintainability  and  health  management  performance- 
based  requirements  had  not  been  demonstrated  as  required  by  the  performance- 
based  specification. 
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Purchasing  (7.4) 

UTAS  was  not  flowing  down  all  applicable  F-35  Program  unique  quality  assurance 
requirements  to  its  suppliers.  UTAS  purchase  orders  did  not  include  Lockheed 
Martin  supplier  quality  assurance  requirements,  such  as  quality  management  system 
requirements,  calibration  requirements,  and  Government  oversight  requirements. 
We  also  noted  numerous  deficiencies  with  UTAS'  review  and  approval  of  manufacturing 
plans.  UTAS  did  not  maintain  required  monthly  supplier  quality  assurance  metrics; 
therefore,  supply  chain  risks  were  not  identified. 

We  noted  inconsistencies  in  the  information  contained  in  the  UTAS  SAP10  system  versus 
the  information  residing  in  the  supplier  quality  assurance  database.  The  process 
for  supplier  rating,  certification,  and  decertification  did  not  follow  UTAS  procedures 
for  procuring  parts  from  qualified  and  authorized  suppliers.  Insufficient  supplier 
management  practices  may  produce  defective  parts  that  go  undetected. 

Product  and  Service  Provision  (7.5) 

Control  of  Production  (7.5.1).  We  documented  that  the  UTAS  master-tooling  list 
had  not  been  updated  and  was  inaccurate.  The  calibration  vendor  for  UTAS  identified 
199  UTAS  tools  in  its  calibration  service  purchase  order;  however,  UTAS’  master-tooling 
list  identified  only  128.  In  addition,  many  tools  on  the  UTAS  production  floor  including 
the  acceptance  testing  fixtures  and  numerous  small  tools  and  gauges  showed  no  evidence 
of  inspection  by  the  quality  assurance  organization  prior  to  use  in  production. 

UTAS  was  not  following  procedures  for  FAIs.  Specifically,  UTAS  did  not  ensure  that  F-35 
landing  gear  system  FAI  data  packages  were  consistently  reviewed  and  approved  and 
contained  the  necessary  supporting  data.  FAI  is  a  process  used  to  verify  and  document 
all  essential  characteristics  on  one  of  the  first  production  components  and  provides 
confidence  that  production  processes,  documentation,  and  tooling  are  sufficient  to 
begin  production.  Several  FAI  packages  were  categorized  as  "FAI  Not  Complete,"  with  no 
corresponding  data  to  explain  why  they  had  not  been  completed. 

Preservation  of  Product  (7.5.5).  UTAS  procedures  for  FOD,  ESD,  and  LSLM  did  not 
ensure  product  protection;  personnel  were  also  not  always  implementing  them  correctly. 
In  one  finding,  FOD-controls  were  not  adequate  to  prevent  FOD  from  being  introduced 
into  the  product.  Examples  of  FOD  objects  noted  in  the  F-35  assembly  area  include  soda 
bottles  on  tool/storage  racks,  a  loose  protective  cap,  residual  tape,  a  small  plastic  bag, 


10  SAP  is  a  German  corporation  that  develops  business  enterprise  software. 
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a  part  identification  label,  a  cigar,  and  sunflower  seeds.  We  noted  that  UTAS  did  not 
have  an  ESD  workstation  or  instructions  for  handling  ESD-sensitive  components  in  the 
production  area.  The  UTAS  F-35  product  line  contains  ESD-sensitive  components  such  as 
the  brake  control  unit  and  the  pressure  transducer.  We  also  identified  several  examples  of 
inappropriately  labeled  LSLMs  with  missing  expiration  dates.  Examples  included  O-ring 
lubricant,  sealing  compound,  red  torque  lacquer,  and  leak  detection  solution. 

Control  of  Monitoring  and  Measurement  Equipment  (7.6) 

We  documented  several  findings  related  to  the  control  of  monitoring  and  measurement 
equipment.  We  found  a  brake  positioning  transducer  that  was  calibrated,  but  the 
inspection  report  failed  to  provide  the  actual  calibration  values;  therefore,  it  could  not 
be  determined  if  the  tool  was  capable  of  making  reliable  measurements.  On  a  transducer 
jig,  the  flatness/parallelism  requirement  was  not  verified  as  part  of  the  inspection 
procedure  and  was  not  recorded  in  the  report.  Another  finding  documented  that  the 
tool  identification  information  for  torque  operations  was  not  recorded  on  multiple  shop 
travelers  as  required  for  traceability.  Several  additional  findings  documented  deficiencies 
with  the  inventory  list,  identification  methods,  and  recordkeeping.  Although  there  were 
processes  to  control  monitoring  and  measurement  equipment,  the  findings  indicate  that 
improvements  are  necessary  to  produce  a  reliable  product. 

Monitoring  and  Measurement  (8.2) 

Internal  Audit  (8.2.2).  The  assessment  documented  one  major  finding  related  to  internal 
audits.  No  product  audits  have  been  performed  on  F-35  landing  gear  assemblies  at  the 
UTAS  Fort  Worth  facility  to  ensure  hardware  meets  end-item  requirements.  Failure  to 
complete  product  audits  as  planned  increases  risk  of  deficient  processes. 

Control  of  Nonconforming  Product  (8.3) 

We  found  that  nonconforming  hardware  was  dispositioned  by  unauthorized  personnel 
instead  of  the  required  engineering  representative,  and  did  not  go  through  the  material 
review  process.  In  addition,  one  of  the  nonconforming  items  we  evaluated  could  not 
be  located. 

UTAS  was  not  tracking  corrective  actions  to  closure  for  product  nonconformances  and 
variances;  therefore,  UTAS  was  unaware  of  the  status  of  these  corrective  actions.  In 
addition,  we  found  that  UTAS  was  not  performing  root  cause  and  corrective  action  for 
scrapped  F-35  parts  and  was  not  submitting  variances  and  scrap  dispositions  to  DCMA  as 
required.  Insufficient  corrective  action  and  lack  of  Government  oversight  increases  the 
risk  of  additional  nonconforming  items  affecting  product  reliability  and  program  cost. 
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Lockheed  Martin  Program  Management  at  UTAS 

We  reviewed  the  effectiveness  of  Lockheed  Martin  program  management  as  part  of 
our  assessment  at  UTAS.  Lockheed  Martin  flowed  down  the  F-35  Brake/Skid  Control 
System  performance-based  specification  with  incomplete  and  conflicting  specification 
requirements.  We  also  found  at  the  time  of  our  assessment  that  Lockheed  Martin  had  not 
funded  root  cause  and  corrective  action  analysis  for  failed  UTAS  hardware.  Failed  parts 
and  detailed  information  of  the  field  failure  was  not  always  provided  back  to  UTAS  for 
analysis;  therefore,  UTAS  was  unable  to  determine  root  cause. 

We  found  that  there  was  no  evidence  that  Lockheed  Martin  had  ever  provided 
"concurrence  in  classification"  for  minor  engineering  changes  submitted  by  UTAS.  UTAS 
personnel  indicated  that  after  30  days,  they  assumed  that  Lockheed  Martin  agreed  with 
the  classification.  Lack  of  Lockheed  Martin  agreement  with  UTAS-initiated  minor  changes 
can  result  in  changes  being  incorporated  and  implemented  without  the  proper  review 
and  approval. 

DCMA  Oversight 

We  found  that  DCMA  Cleveland  did  not  provide  the  required  Government  oversight  of 
UTAS  waivers,  deviations,  and  nonconformances  as  required  by  the  LOD.  Additionally, 
DMCA  Cleveland  was  not  delegating  nonconforming  material  review  responsibilities  to 
the  appropriate  DCMA  offices  located  at  UTAS  suppliers. 

UTAS  Assessment  Summary 

We  found  a  lack  of  disciplined  process  controls  for  FOD,  ESD,  and  LSLM.  This  could  result 
in  nonconforming  flight  hardware  or  latent  defects. 

Several  key  design  and  development  tasks  were  not  completed.  Although  the  program 
has  transitioned  to  LRIP,  UTAS  had  not  received  Lockheed  Martin's  approval  for 
10  development  specifications  and  ICDs.  UTAS  also  had  not  verified  technical  performance 
requirements  testing  prior  to  the  start  of  LRIP. 

UTAS  did  not  conduct  internal  audits  at  the  Fort  Worth  facility,  and  was  not  tracking 
corrective  actions  to  closure  for  product  nonconformances  and  variances.  The  lack  of 
an  internal  auditing  program  and  disciplined  corrective  action  process  could  result  in 
quality  assurance  issues  and  additional  cost  growth  to  the  program. 
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Overall  Findings  and  Recommendations 

The  F-35  Program  did  not  sufficiently  implement  or  flow  down  technical  and  quality 
management  system  requirements  to  prevent  the  fielding  of  nonconforming  hardware 
and  software.  This  could  adversely  affect  aircraft  performance,  reliability,  maintainability, 
and  ultimately  program  cost.  Lockheed  Martin  and  its  subcontractors  did  not  follow 
disciplined  AS9100  Quality  Management  System  practices,  as  evidenced  by  363  findings, 
which  contained  719  issues. 


A.  Insufficient  Rigor  in  Design,  Manufacturing,  and 
Quality  Assurance  Processes 

The  F-35  JPO,  Lockheed  Martin,  and  its  subcontractors  were  not 
ensuring  that  the  necessary  quality  assurance  process  controls 
and  disciplines  were  in  place  to  produce  a  consistent  and  reliable 
product.  This  lack  of  process  discipline  and  attention  to  detail 
creates  an  elevated  risk  of  delivering  nonconforming  aircraft  to 
the  warfighter. 

The  root  cause  of  nearly  half  of  the  findings  was  the  lack  of  adherence  to  established 
processes  and  procedures  or  insufficient  detail  in  documentation.  The  major  process 
discipline  issues  were  in  the  following  areas. 

•  Planning  for  Product  Realization  -  Production  planning  lacked  the 
appropriate  level  of  detail,  design  changes  were  uncontrolled  or  unapproved, 
and  system-level  plans  were  not  maintained.  As  a  result,  documentation 
did  not  always  match  the  processes  and  equipment  required  to  perform  the 
manufacturing  operation. 

•  Design  and  Development  -  Contractor  personnel  were  using  insufficient  or 
unapproved  test  plans  and  procedures.  Systems  engineering  technical  review 
entrance  and/or  exit  criteria  were  not  met.  The  criteria  ensure  the  program 
is  ready  to  proceed  to  the  next  production  milestone  or  test  event. 

•  Software  Development  -  In  some  cases,  software  design  and  development 
activities  lacked  definition  and  rigorous  adherence  to  documented 
procedures  and  processes,  resulting  in  software  development  requirements 
not  implemented,  traceable,  and  verified. 
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•  Manufacturing  Operations  -  Contractor  personnel  were  not  following 
documented  manufacturing  and  assembly  process  instructions  necessary 
to  ensure  conformance  to  end-item  requirements,  product  protection,  and 
material  control. 

•  Production  Tooling  -  Uncontrolled  and  unproven  tooling  was  in  use  on  the 
production  floor,  resulting  in  product  nonconformances. 

•  Calibration  of  Measurement  Systems  -  Multiple  sites  had  inadequate 
calibration  management  systems.  Contractors  were  not  performing  out- 
of-tolerance  condition  analyses,  using  sufficient  calibration  procedures, 
and  removing  expired  calibrated  equipment  from  use.  Accurate  test  and 
measurement  equipment  is  necessary  to  provide  accurate  measurements  that 
determine  product  acceptability. 

•  Engineering  Change  Management  -  Lockheed  Martin  was  not  reviewing 
and  approving  all  subcontractor  submittals  for  variances  and  engineering 
change  requests  for  which  Lockheed  Martin  is  the  sole  design  authority. 
Additionally,  Lockheed  Martin  was  not  providing  concurrence  for  engineering 
change  classifications  for  subcontractor  engineering  changes. 

Recommendation  A— Management  Comments  and 
Our  Response 

We  recommend  that  the  Joint  Program  Office: 

1.  Ensure  compliance  with  AS9100  throughout  the  F-35  supply  chain. 

Joint  Program  Office  Comments 

JPO  agreed  and  stated: 

AS9100  (Quality  Management  Systems  -  Requirements  for 
Aviation,  Space,  and  Defense  Organizations]  is  an 
F-35  contractual  requirement.  The  prime  contractor 

(Lockheed  Martin  (LM]]  is  AS9100  certified  and  has  flowed 
down  this  quality  requirement  to  its  supply  chain.  AS9100 
certification  is  granted  by  a  third  party  accredited  entity. 

Periodic  reviews  are  held  by  that  third  party  entity  to  maintain 
certification.  Annual  self  audits  are  conducted  by  both  the 
prime  contractor  and  its  suppliers  to  monitor  compliance 
within  their  respective  facilities.  The  Government  has  had 
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limited  insight  to  results  of  these  self  audits,  however,  DCMA 
performs  onsite  process  reviews  as  part  of  its  oversight 
activity.  DCMA  at  Lockheed  Martin  Fort  Worth  reorganized  in 
February  2013  to  provide  more  focus  and  inspection 
emphasis  to  ensure  AS9100  compliance  and  a  quality 
product.  Additionally,  JPO  reviews  AS9100  certification 
status  of  critical  suppliers  during  their  annual  Production 
Readiness  Reviews  [PRR],  These  reviews  serve  as  a  LRIP 
risk  management  tool  conducted  with  the  prime  contractor 
and  selected  major  critical  suppliers,  covering  Technical, 
Production  and  Sustainment  requirements.  Suppliers  must 
demonstrate  consistent  adherence  of  product  compliant  to 
contractual  [AS9100]  and  applicable  statutory  and  regulatory 
requirements.  Findings  of  this  IG  audit  have  caused  both 
contractor  and  Government  entities  to  adjust  the  frequency 
and  scope  of  independent  audits  to  ensure  greater  insight  to 
AS9100  compliance. 

Our  Response 

The  comments  from  JPO  are  responsive,  and  the  actions  taken  appear  to  meet  the 
intent  of  the  recommendation. 

2.  Ensure  that  Lockheed  Martin  approves  all  design  and  material  review 
board  changes  and  variances  with  Government  concurrence. 

Joint  Program  Office  Comments 

JPO  agreed  and  stated: 

Current  F-35  Program  plans  (such  as  the  Configuration 
Management  Plan]  require  that  all  major  design  changes 
and  variances  for  production  aircraft  receive  Government 
concurrence.  The  Government  maintains  limited  insight 
of  the  contractor's  Material  Review  Board  (MRB)  process. 
Subsequently,  the  Government  has  reprioritized  and  refined  its 
engagement  in  that  process  to  ensure  improved  effectiveness. 

For  nonconformances  that  require  MRB  disposition,  JPO 
delegates  the  approval  of  Class  II  [minor]  changes  to  DCMA 
and  reviews  all  Class  I  changes  that  affect  form,  fit,  or 
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function.  A  Class  I  change  request  is  dispositioned  through 
the  JPO  Change  Review  Board  [JCRB],  which  performs  a 
technical  and  business  assessment  and  provides  a  written 
recommendation  to  the  Joint  Configuration  Control  Board 
(JCCBJ.  With  the  input  of  major  stakeholders,  the  JCCB 
adjudicates  the  recommended  change  and  provides  that 
decision  to  Contracts,  who  notifies  the  Prime  Contractor  to 
proceed  in  incorporating  the  change. 

Any  subcontractor  submittal  for  variances  and  engineering 
change  requests,  for  which  Lockheed  Martin  is  the  sole 
design  authority,  must  be  go  through  Lockheed  Martin's 
change  control  boards  for  review  and  approval.  Furthermore, 

Lockheed  Martin  must  provide  concurrence  for  engineering 
changes  relating  to  subcontractor  owned  designs.  Various 
configuration  management  checks  and  balances  (such  as 
Physical  and  Functional  Configuration  Audits]  are  in  place  to 
ensure  control. 

Our  Response 

The  comments  from  JPO  are  responsive.  The  actions  taken  appear  to  meet  the  intent 
of  the  recommendation;  however,  the  DoD  OIG  suggests  that  the  JPO  and 
Lockheed  Martin  maintain  cognizance  of  class  II  changes. 

3.  Perform  process  proofing  of  all  critical  processes  to  include  first 
article  inspections. 

Joint  Program  Office  Comments 

JPO  partially  agreed  and  stated: 

The  Government  does  not  have  the  resources  to  perform 
process  proofing  of  all  critical  processes.  Responsibility  and 
accountability  rests  on  the  prime  contractor,  with  oversight 
from  the  Government.  Process  proofing  and  First  Article 
Inspections  are  Program  Plan  and  contractual  requirements. 

The  Government  performs  audits  of  process  compliance  to 
ensure  the  contractor  is  controlling  critical  operations.  The 


58  |  DODIG-2013-140 


Overall  Findings  and  Recommendations 


Government  closely  monitors  scrap,  rework,  and  repair  to 
ensure  the  contractor  is  conducting  proper  root  cause  analyses 
to  implement  appropriate  corrective  actions.  The  Government 
also  monitors  the  quality  of  delivered  product  to  ensure  control 
of  escaped  defects. 

JPO  requires  the  contractor  to  perform  First  Article  Inspections 
(FAIJ  and  process  proofing  as  part  of  their  implementation 
of  AS9100  and  the  F-35  Program  Quality  Management  Plan. 

Over  27,500  original  baseline  FAIs  have  been  completed  and 
approximately  400  more  delta  FAIs  will  be  performed  for 
parts  due  to  concurrency,  producibility,  process  changes, 
etc.  In  addition,  the  prime  contractor  has  implemented 
variation  management,  which  is  an  advanced  quality  system 
technique  that  focuses  on  defect  prevention  and  continuous 
improvement  through  the  identification  of  key  product  and 
process  characteristics.  This  allows  greater  control  of  the 
manufacturing  process. 

Our  Response 

Although  we  understand  the  concern  for  Government  resources,  we  still  recommend 
that  JPO  ensure  process  proofing  of  all  critical  processes.  A  widely  accepted  industry 
handbook  on  quality,  Juran's  Quality  Handbook,  defines  critical  properties  as:  "Critical 
properties  are  those  properties  for  which  the  made-product  total  variability  is  so 
wide  as  to  cause  persistent  difficulty  in  meeting  current  or  near-term  anticipated 
customer  needs."  The  Government  must  ensure  that  the  contractor  performs  some 
level  of  process  proofing  to  prevent,  as  Juran’s  Quality  Handbook  states,  "variability 
in  critical  product  characteristics,  which  is  the  enemy  of  quality  of  conformance." 

As  evidenced  by  the  findings,  process  proofing  had  not  been  accomplished  to  the 
level  to  ensure  product  repeatability.  It  is  the  DoD  OIG's  position  that  without 
process  proofing  of  all  critical  processes,  quality  assurance  may  not  be 
achieved.  Therefore,  we  request  further  comments  from  the  JPO  in  response  to 
the  final  report. 

4.  Modify  its  contracts  to  include  a  quality  escape  clause  to  ensure  the 
Government  does  not  pay  for  nonconforming  product. 
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Joint  Prog ra m  Offi ce  Comments 

JPO  agreed  and  stated: 

JPO  concurs  that  greater  incentives  should  be  taken  to 
preclude  the  cost  of  poor  quality.  Major  non-conformances 
that  do  not  meet  engineering  specification  are  identified 
and  adjudicated  for  acceptance  or  rejection  with 
considerations/withholds  placed  on  a  variance.  Minor 
non-conformances  are  reworked,  repaired,  or  used  as  is. 

The  program  has  transitioned  to  fixed  price  type  contacts 
with  a  0/100  share  ratio  and  clauses  will  prescribe 
to  applicable  FAR  [Federal  Acquisition  Regulation] 
requirements.  Along  with  considerations/withholds  on 
non-conformances,  this  contract  structure  will  facilitate 
greater  incentive  by  the  contractor  to  provide  quality 
assurance  in  order  to  maintain  cost  and  schedule  obligations 
without  over  running  their  negotiated  budget.  The 
contractor  will  absorb  a  level  of  cost  for  poor  quality.  In 
addition,  the  JPO  continues  to  establish  quality  performance 
targets  with  commitments  from  JPO  and  LM  senior 
leadership.  These  targets  are  based  upon  continuous 
learning  in  fabrication  and  assembly  operations,  corrective 
action  implementation,  in  addition  to  product  and 
process  improvements  as  a  result  of  affordability  and 
concurrency  changes. 

Our  Response 

The  comments  from  JPO  are  responsive,  and  the  actions  taken  appear  to  meet  the 
intent  of  the  recommendation. 

5.  Assess  the  impacts  and  risks  to  all  delivered  aircraft  for  all  findings. 

Joint  Program  Office  Comments 

JPO  agreed  and  stated: 

The  impacts  and  risks  of  all  findings  were  assessed  as 
part  of  the  Corrective  Action  Request  (CARJ/Corrective 
Action  Plan  (CAP]  process.  When  a  CAR  was  written,  the 
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F-35  fleet  was  assessed  for  impact.  Concurrence  on  each  CAP 
considered  effectivity,  safety,  and  contract  elements.  This 
determined  priority  and  timing  required  to  implement  the 
corrective  action  into  the  affected  aircraft  as  appropriate. 

Our  Response 

The  comments  from  JPO  are  responsive,  and  the  actions  taken  appear  to  meet  the  intent 
of  the  recommendation. 

B.  Critical  Safety  Item  Program  Not  Implemented 

The  F-35  JPO  did  not  include  CSI  requirements  on  the  Lockheed 
Martin  prime  contract  as  required  by  Public  Law  and  DoD 
policy.  Aircraft  CSIs  were  not  identified  and  critical  parts  were 
not  manufactured  with  CSI  controls  and  verifications.  Delivered 
aircraft  may  pose  an  increased  safety  of  flight  risk  due  to  the  lack 
of  critical  process  control. 

JPO  has  not  flowed  down  a  contractual  requirement  to  Lockheed  Martin  (and 
subsequently,  its  subtier  suppliers]  to  implement  a  CSI  management  program  in 
accordance  with  Public  Law  and  DoD  policy.  A  CSI  is  a  part,  assembly,  or  support 
equipment  whose  failure  could  cause  loss  of  life,  permanent  disability  or  major 
injury,  loss  of  a  system,  or  significant  equipment  damage.  Special  attention  should  be 
paid  to  CSIs  to  prevent  the  potential  catastrophic  or  critical  consequences  of  failure. 
CSIs  require  special  handling,  engineering,  manufacturing,  and  inspection 
documentation  to  control  and  ensure  safety  of  flight. 

Public  Law  108-136,  Section  802,  "Quality  control  in  procurement  of  aviation  CSIs  and 
related  services,”  requires  DoD  to  prescribe  a  quality  control  policy  for  the  procurement 
of  aviation  CSIs.  Joint  Service  CSI  Instruction,  "Management  of  Aviation  Critical  Safety 
Items,”  implements  the  DoD  CSI  program  and  establishes  the  policies,  procedures, 
and  responsibilities  to  manage  CSI.  The  Joint  Aeronautical  Commanders  Group,  "JACG 
Aviation  Critical  Safety  Item  Handbook,"  implements  the  policies  in  the  Joint  Service  CSI 
Instruction  and  describes  the  technical  and  quality  assurance  requirements  for  a  Prime 
Manufacturer  CSI  program.  Without  a  CSI  program,  there  is  an  increased  safety  risk  to  the 
aircraft  and  warfighter. 
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Recommendation  B— Management  Comments  and 
Our  Response 

We  recommend  that  the  Joint  Program  Office: 

1.  Implement  an  aviation  critical  safety  item  program  that  meets  the 
requirements  of  Public  Law  and  DoD  policy,  which  would  include 
flow  down  of  requirements  for  a  critical  safety  item  program  to 
Lockheed  Martin  and  its  subcontractors. 

Joint  Prog ra m  Offi ce  Comments 

JPO  agreed  and  stated: 

JPO  is  working  with  the  prime  contractor  in  developing 
a  Request  for  Proposal  (RFP]  for  a  two-phased  CSI 
Non  Recurring  Engineering  (NRE]  approach.  This  will  require 
the  development  of  comprehensive  CSI  requirements  which 
will  be  added  to  the  SDD  contract's  Statement  of  Work.  These 
requirements  will  address  the  identification  of  critical 
characteristics  that  enable  compliance  at  the  prime  contractor 
and  throughout  its  supply  base.  Once  the  SDD  NRE  efforts 
are  completed,  the  full  scope  of  the  F-35  CSI  program  will 
be  included  in  future  production  and  sustainment  contracts. 
Commencement  of  certain  initial  recurring  tasks  will  be 
implemented  using  the  LRIP  6  contract.  In  the  interim,  the 
contractors'  integrity  and  quality  programs  are  providing 
assurance  for  continued  aviation  safety. 

Our  Response 

The  comments  from  JPO  are  responsive,  and  the  actions  appear  to  meet  the  intent  of 
the  recommendation. 

2.  Assess  the  impacts  and  risks  to  all  delivered  aircraft  for  critical  safety 
item  deficiencies. 
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Joint  Prog ra m  Offi ce  Comments 

JPO  agreed  and  stated: 

Deficiencies  and  impacts  to  critical  parts  are  identified  and 
assessed  during  quality  assurance  practices  and  testing 
operations  as  part  of  the  SDD  design  and  product  verification 
processes.  If  safety  issues  and  limitations  are  discovered, 
changes  are  initiated  and  incorporated  as  warranted  to  the  test 
and  production  aircraft.  Initial  and  continued  airworthiness 
is  a  major  focus  of  the  F-35  Class  Desk,  Airworthiness  Team, 

Integrated  Test  Team,  and  Integrated  Product  Teams.  The 
SDD  test  aircraft  have  more  than  5,000  flight  hours.  The 
combined  F-35  fleet  has  accumulated  more  than  7,000  flight 
hours.  This  is  not  to  say  that  there  are  no  significant  issues  or 
discoveries.  Rather,  there  is  an  upfront  understanding  whether 
any  new  discovery  poses  an  increased  safety  risk.  If  necessary, 
proper  actions  are  taken,  such  as  suspending  flight  operations, 
directing  inspections,  adding  flight  limitations  or  restrictions, 
and  directing  hardware  or  software  modifications  to  mitigate 
the  safety  risks. 

Our  Response 

The  comments  from  JPO  do  not  meet  the  intent  of  the  recommendation. 
Per  our  findings  the  JPO  did  not  create  a  CSI  program  in  accordance  with 
Public  Law  108-136,  Section  802.  The  DoD  OIG  desires  the  identification 
and  confirmation  from  the  Engineering  Support  Activity  that  all  safety  risks  for 
aircraft  manufactured  and  delivered  to  the  Government  using  the  methodology 
identified  in  the  CSI  handbook  is  complete.  Therefore,  we  request  further 
comments  from  the  JPO  in  response  to  the  final  report. 

C.  Requirements  Not  Flowed  Down 

JPO  and  Lockheed  Martin  did  not  ensure  that  all  essential  quality 
assurance  and  technical  requirements  were  flowed  down  to 
the  supply  chain.  This  resulted  in  suppliers  providing  critical 
F-35  products  that  may  not  meet  intended  performance  and 
reliability  requirements. 
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We  noted  a  significant  number  of  findings  in  this  area.  These  include  the  following. 

•  First  Article  Inspection  -  Lockheed  Martin  flowed  down  conflicting 
requirements  for  FAIs  to  its  suppliers.  The  lack  of  definitive  FAI 
requirements  was  affecting  the  suppliers'  ability  to  qualify  its  processes 
and  ensure  delivery  of  compliant  product. 

•  Configuration  Control  -  Lack  of  flow  down  and  communication  between 
Lockheed  Martin  and  suppliers  was  causing  unclear  configuration 
management  and  status  accounting  requirements.  Configuration  issues  at 
suppliers  were  resulting  in  products  with  unknown  configurations. 

•  Design  and  Development  Requirements  -  Lockheed  Martin  did  not  flow 
down  all  the  necessary  design  and  development  requirements  and  ensure 
requirements  were  verified  and  traceable.  This  impacted  the  suppliers'  ability 
to  develop  products  that  meet  expected  performance  requirements. 

•  Diminishing  Manufacturing  Supply  and  Materiel  Shortages  -  JPO  had  not 

funded  Lockheed  Martin  to  implement  a  DMSMS  program.  As  a  result,  the 
program  was  not  proactively  forecasting  for  part  obsolescence. 

Recommendation  C— Management  Comments  and  Our 
Response 

We  recommend  that  the  Joint  Program  Office  perform  technical  and  quality 
assurance  requirement  flow  down  and  verification  throughout  the  F-35 
supply  chain. 

Joint  Prog ra m  Offi ce  Comments 

JPO  partially  agreed  and  stated:: 

The  Government  does  not  have  the  responsibility  or  resources 
to  perform  requirement  flow  down  verification  throughout  the 
prime  contractor's  entire  supply  chain.  The  prime  contractor 
is  responsible  and  accountable,  with  oversight  from  the 
Government.  The  Government  performs  spot  checks  and 
conducts  surveillance  to  ensure  the  contractor  is  flowing 
down  requirements  to  its  subcontractors.  The  Government 
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monitors  this  closely  through  participation  in  Design  Reviews, 

Production  Readiness  Reviews,  Structural  Equivalency 
Reviews,  Program  Management  Reviews,  production  floor 
inspections,  configuration  audits,  etc.  On  site  surveillance  is 
primarily  performed  by  DCMA  to  ensure  the  contractor  and  its 
subcontractors  are  complying  with  contractual  requirements. 

Our  Response 

The  DoD  OIG  does  not  agree  that  the  Government  does  not  have  the  responsibility 
to  ensure  that  the  prime  contractor  and  the  supply  chain  meet  requirements.  In 
accordance  with  DoDD  5000.01,  section  3.5,  the  Program  Manager  has  the  responsibility 
for  and  authority  to  accomplish  program  objectives  for  development,  production,  and 
sustainment  to  meet  the  user's  operational  needs.  The  intent  of  our  recommendation 
is  for  JPO  to  ensure  that  technical  and  quality  assurance  requirement  flow  down  occurs 
by  establishing  the  appropriate  contractual  requirements;  such  as  having  the  prime 
contractor  create  a  database  showing  each  supplier  and  all  applicable  performance 
and  quality  assurance  requirements.  Contractual  deliverables  such  as  database  access 
and  various  monthly  metrics  would  provide  the  Government  additional  confidence 
that  this  has  occurred.  The  lack  of  a  requirement  flow  down  process  to  date  has 
hampered  the  program's  understanding  of  supplier  conformance  to  performance  and 
quality  contractual  requirements.  Therefore,  we  request  further  comments  from  JPO  in 
response  to  the  final  report. 

D.  Ineffective  Quality  Assurance  Organization 

JPO's  quality  assurance  organization  did  not  have  the  appropriate 
resources  and  authority  to  effectively  manage  DoD's  largest 
acquisition  program.  The  lack  of  a  strong  and  effective  quality 
assurance  organization  contributed  to  the  program's  cost, 
schedule,  and  performance  issues. 

DoDD  5000.01,  "The  Defense  Acquisition  System,"  designates  authority  and  responsibility 
to  the  program  manager  within  JPO  to  accomplish  program  objectives  for  development, 
production,  and  sustainment  to  ensure  that  reliable  systems  are  delivered  to  the 
warfighter.  DoDD  5000.01  goes  on  to  state  that  the  program  manager,  "shall  be  the  single 
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point  of  accountability  for  accomplishing  program  objectives  for  total  life-cycle  systems 
management."  As  evidenced  by  our  assessment  that  identified  363  findings,  JPO  appeared 
to  rely  on  Lockheed  Martin  and  DCMA  to  identify,  report,  and  address  quality  assurance 
issues.  This  indicates  a  lack  of  quality  assurance  and  technical  expertise  within  JPO  to 
recognize  F-35  supply  chain  issues. 

Furthermore,  the  Lockheed  Martin  Aeronautics  quality  assurance  leadership  team 
reports  to  the  Lockheed  Martin  Aeronautics  operations  leadership  team  and  does  not 
have  the  organizational  independence  or  inherent  authority  to  enforce  quality  assurance 
internally  and  throughout  the  F-35  supply  chain.  It  should  be  noted  that  Lockheed  Martin 
Space  Division  has  an  independent  quality  assurance  organization  that  does  not  report 
to  operations. 

Recommendation  D— Management  Comments  and 
Our  Response 

We  recommend  that  the  Joint  Program  Office  establish  an  independent  quality 
assurance  organization,  which  has  the  authority  and  resources  to  enforce  the 
AS9100  standard  and  F-35  product  quality. 

Joint  Program  Office  Comments 

JPO  disagreed  with  the  recommendation.  JPO  stated: 

JPO  acknowledges  opportunities  to  improve  management  of 
Quality  performance  and  the  benefits  of  realignment  within 
the  F-35  Program  organizational  structure  to  better  support 
transition  from  Low  Rate  Initial  Production  to  Full  Rate 
Production.  These  efforts  have  commenced  to  include  an 
increase  in  resources  and  realignment  of  the  Quality  Team 
to  report  to  program  leadership.  The  JPO  non-concurs  with 
establishing  an  independent  quality  assurance  organization. 

DCMA  performs  the  role  of  the  independent  quality 
assurance  organization  for  the  F-35  and  other  DoD  programs. 

DCMA  conducts  process  surveillance  against  the  AS9100 
standard  and  enforces  F-35  product  quality  requirements. 
Additionally,  the  JPO  Quality  Team  is  staffed  from  two 
independent  organizations;  Air  Force  Life  Cycle  Management 
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Center  (AFLCMC]  and  Naval  Air  Systems  Command  (NAVAIR], 

While  serving  on  the  JPO  Quality  Team,  these  individuals 
retain  their  responsibility  to  invoke  their  independent 
AFLMC  and  NAVAIR  leadership  in  the  event  the  JPO  does  not 
properly  address  quality  concerns. 

Our  Response 

We  disagree  with  JPO's  approach  because  DCMA,  AFLCMC,  and  NAVAIR  are  not 
accountable  for  program  quality  assurance  goals.  In  accordance  with  DoDD  5000.01, 
section  3.5,  the  Program  Manager  has  the  responsibility  for  and  authority  to 
accomplish  program  objectives  for  development,  production,  and  sustainment  to 
meet  the  user's  operational  needs.  Quality  assurance  is  inherently  a  Government 
program  office's  responsibility.  As  evidenced  by  our  findings,  JPO  did  not  enforce 
F-35  product  quality  requirements.  DCMA  uses  a  "riskbased  surveillance  approach” 
to  apply  resources  across  the  DoD.  Based  on  the  findings  from  our  assessment,  DCMA 
did  not  apply  the  appropriate  resource  throughout  the  supply  chain.  JPO  must  ensure 
that  Government  resources  are  integrated  into  the  program,  efficiently  applied,  and 
understand  the  system  criticality  of  each  aircraft  component.  It  should  also  ensure 
that  design,  test,  and  manufacturing  issues  are  addressed  with  long-term,  permanent 
solutions.  The  DoD  OIG  seeks  an  effective  quality  assurance  organization  that  meets 
these  expectations.  Therefore,  we  request  further  comments  from  JPO  in  response  to 
the  final  report. 

E.  Ineffective  DCMA  Oversight 

DCMA  was  not  performing  quality  assurance  oversight 
commensurate  with  product  criticality.  Insufficient  written 
direction  from  JPO  coupled  with  inadequate  execution  from 
DCMA  resulted  in  ineffective  Government  oversight  of  the 
F-35  Program. 

In  accordance  with  the  F-35  Program  memorandum  of  agreement,  DCMA  has 
been  delegated  the  responsibly  of  final  acceptance  of  each  production  article.  The 
memorandum  states  "DCMA  will  issue  LODs  to  support  Contract  Management  Offices  for 
key  JSF  subcontractors  as  defined  by  DCMA  and  based  on  the  analysis  of  subcontractor 
performance  data  and  risk.”  In  addition,  DCMA  will  analyze  contractor-earned  value 
data  and  provide  an  assessment  of  program  risk  based  on  cost,  schedule,  and  technical 
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performance.  DCMA  will  also  maintain  the  Joint  Surveillance  Plan  (between  DCMA  and 
Lockheed  Martin],  The  plan  provides  the  details  for  accomplishing  system  surveillance 
and  maintenance. 

Although  there  was  a  memorandum  of  agreement  between  JPO  and  DCMA,  it  was 
inadequate  for  the  complexity  of  the  F-35  Program.  The  memorandum  of  agreement 
did  not  provide  any  specifics  to  ensure  adequate  oversight  of  the  F-35  supply  chain. 
DCMA  Fort  Worth  did  not  issue  letters  of  delegation  to  cognizant  DCMA  offices  at  key 
F-35  suppliers  based  on  the  analysis  of  supplier  performance  data  and  risk.  DCMA  did 
not  consistently: 

•  develop  or  implement  risk-based  surveillance  plans  in  accordance  with 
DCMA  policies, 

•  participate  in  material  review  board  activities, 

•  adequately  conduct  oversight  activities,  and 

•  identify  and  report  oversight  activities  performed  at  lower  tier  suppliers. 

Recommendation  E— Management  Comments  and 
Our  Response 

1.  We  recommend  that  the  Joint  Program  Office: 

a.  Revise  the  Defense  Contract  Management  Agency  memorandum  of 
agreement  to  provide  explicit  surveillance  criteria  for  mission-critical 
hardware  and  software,  to  include,  but  not  limited  to,  material  review, 
mandatory  government  inspection,  process  audit,  product  acceptance. 

Joint  Prog ra m  Offi ce  Comments 

JPO  agreed  and  stated: 

The  Memorandum  of  Agreement  [MOA],  annex  A-P,  dated 
February  2011,  between  the  JPO  and  DCMA  is  being  updated 
to  reflect  Program  objectives  requiring  DCMA  support  in  the 
following  areas:  Management  of  contractor  Quality  Process 
implementation,  control,  maturity,  and  verification;  Material 
Review/Change  Board  Process  authority  and  related  task 
requirements,  Foreign  Object  Damage  [FOD]  management, 
Government  Flight  Representative  surveillance,  Safety  of 
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Flight  inspection  requirements,  contractual  Corrective  Action 
Request  notification,  Air  system  acceptance  (DD250J,  Supply 
Chain  Management,  etc.  However  specific  surveillance 
activities  and  priorities  are  articulated  via  other  tools,  such 
as  Quality  Assurance  Letters  of  Inspection  (QALI’sJ. 

It  is  important  to  note  that  the  MOA  update,  though  crucial  in 
shaping  organizational  roles  and  responsibilities  that  address 
the  findings  in  this  report,  is  only  part  of  the  corrective  action 
process  being  implemented.  Execution  of  DCMA  policies, 
enhanced  communication  between  the  JPO  and  DCMA,  and 
enhanced  inter-DCMA  communications  with  supplier  DCMA 
organizations  all  play  a  role  in  improving  surveillance.  In 
addition  to  the  MOA,  DCMA  utilizes  risk  based  surveillance 
plans  to  support  F-35  contracts.  As  part  of  the  functional 
surveillance  planning  process,  DCMA  determines  what  types 
of  surveillance  techniques  will  be  applied  to  ensure  a  supplier 
has  adequate  quality  processes  in  place  and  meet  all  contractua 
lrequirements.  Risks  are  identified  through  historical 
knowledge  of  the  suppliers  systems  and  processes,  results  of 
surveillance  execution,  analysis  of  DCMA  internal  data,  supplier 
data  and  customer  data. 

Our  Response 

The  comments  from  JPO  are  responsive  and  the  actions  met  the  intent  of 
the  recommendation. 

b.  Ensure  that  Defense  Contract  Management  Agency  is  performing  quality 
assurance  oversight  commensurate  with  product  criticality. 

Joint  Prog ra m  Offi ce  Comments 

JPO  agreed  and  stated: 

DCMA  has  been,  and  continues  to  be,  responsive  to  product 
criticality  while  performing  its  contact  administration 
quality  assurance  functions.  This  includes  optimizing 
quality  assurance  oversight  of  contractors'  critical  processes, 
operations,  and  product.  Product  criticality  is  governed  by 
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Federal  Acquisition  Regulation  derived  surveillance  strategies. 

FAR/DFAR  regulatory  requirements  as  manifest  in  F-35 
contracts  are  also  guided  by  DCMA  policies.  DCMA  policies 
define  risk  considerations  and  thresholds  in  developing 
surveillance  plans.  Quality  specialists  determine  what  types  of 
surveillance  techniques  will  be  applied  to  ensure  a  supplier 
has  adequate  quality  processes  in  place.  Surveillance  activities 
and  data  analysis  allow  DCMA  to  re-evaluate  risk  levels  and 
where  appropriate  adjust  surveillance  plans. 

As  non-conformance  is  identified,  JPO  may  request  special 
surveillance  be  performed  in  other  areas  due  to  severity 
or  whether  they  are  systematic  in  nature.  This  is  captured 
in  other  documents  such  as  QALI’s.  The  DCMA  monthly 
report  also  is  a  means  of  communicating  quality  assurance 
oversight  as  it  pertains  to  production  status.  Continuous 
communication  between  JPO  and  DCMA  to  discuss  and  resolve 
quality  issues  is  critical  to  meeting  Program  objectives. 

Performing  quality  assurance  in  accordance  with  product 
criticality  is  central  to  how  DCMA  conducts  contract  surveillance. 

Our  Response 

The  comments  from  JPO  do  not  meet  the  recommendation  intent.  As  evidenced  by 
the  findings,  DCMA  is  not  providing  sufficient  supply  chain  oversight  commensurate 
with  product  criticality.  The  term  "surveillance,"  as  defined  in  Federal  Acquisition 
Regulation  (FAR)  42.11,  is  broad  and  does  not  specifically  equate  to  government 
verification  or  inspection  of  manufacturing  processes  or  products.  As  an  example, 
the  DCMA  surveillance  for  a  supplier  could  be  nothing  more  than  an  itinerant  DCMA 
person  visiting  a  particular  supplier  every  six  months  with  the  visit  entailing  a  brief 
walk  through  of  the  facility.  Therefore,  it  is  our  position  that  JPO’s  quality  assurance 
organization  should  make  the  final  decision  on  supplier  risk,  and  the  type  and  amount 
of  DCMA  surveillance.  We  request  further  comments  from  the  JPO  in  response  to  the 
final  report. 
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2.  We  recommend  that  the  Defense  Contract  Management  Agency: 

a.  Provide  a  comprehensive  quality  assurance  oversight  plan  for 
Joint  Program  Office  approval  to  be  included  in  the  memorandum 
of  agreement. 

Defense  Contract  Management  Agency  Comments 

DCMA  partially  agreed.  DCMA  stated: 

DCMA  will  ensure  the  MOA,  as  currently  being  updated, 
complements  DCMA  risk  based  surveillance  planning.  The 
DMCA  surveillance  planning  process  is  risk  based  and 
developed  through  historical  knowledge  of  the  suppliers 
systems  and  processes,  results  of  surveillance  execution, 
analysis  of  DCMA  internal  data,  supplier  data  and 
customer  data. 

DCMA  is  also  implementing  a  dedicated  Supply  Chain 
Management  team  at  LMFW  [Lockheed  Martin,  Fort  Worth] 
to  address  the  significant  supply  chain  oversight  challenge  of 
the  F-35  program.  The  focus  of  the  team  will  be  to  evaluate 
LMFWs  supply  chain  processes  and  system  as  well  as  a 
more  focused  emphasis  on  risk  suppliers  throughout  the 
Supply  Chain. 

Our  Response 

The  comments  from  DCMA  do  not  meet  the  recommendation  intent.  As  stipulated 
in  our  E.l.b  response,  it  is  important  for  JPO's  quality  assurance  organization 
to  make  the  final  decision  on  supplier  risk  and  the  type  and  amount  of  DCMA 
oversight.  A  comprehensive  quality  assurance  oversight  plan  would  specifically 
identify  DCMA  oversight  for  critical  assemblies:  inspection,  verification,  and 
process  proofing.  Therefore,  we  request  further  comments  from  the  DCMA  in 
response  to  the  final  report. 

b.  Audit  the  execution  of  the  quality  assurance  oversight  plan  throughout 
the  F-35  supply  chain. 
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Defense  Contract  Management  Agency  Comments 

DCMA  agreed  and  stated: 

DCMA  will  comply  with  its  quality  assurance  oversight  plan. 

Regular  audits  of  supplier  and  prime  DCMA  organizations 
are  crucial  to  maximizing  the  effectiveness  of  risk  based 
surveillance.  Within  DCMA,  Management  Review  Teams  (MRT] 
provide  comprehensive  functional  reviews  on  our  quality 
program.  These  risk-based  audit  efforts,  are  conducted  by 
independent  quality  experts  and  occur  throughout  the  entire 
DCMA  organization  ensuring  coverage  of  the  F-35  supply  chain. 

Continued  communication  of  audit  results  with  our  customers 
and  enhanced  communication  between  DCMA  supplier  CMOs 
will  also  supplement  a  robust  audit  process. 

Our  Response 

The  DCMA  comments  do  not  meet  the  recommendation  intent.  DCMA  states  they 
will  comply  with  its  existing  quality  assurance  oversight  plan,  which  does  not 
appear  to  be  the  comprehensive  F-35  quality  assurance  oversight  plan  sought  in  our 
E.2.a  recommendation.  In  addition,  the  DCMA  audit  and  MRT  process  cited  is  not 
new  and  as  identified  by  our  assessment  was  insufficient.  Therefore,  we  request 
further  comments  from  the  DCMA  in  response  to  the  final  report. 
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Appendix  A 

Scope  and  Methodology 

We  conducted  this  assessment  from  February  2012,  through  July  2013,  in  accordance 
with  the  Council  of  the  Inspectors  General  on  Integrity  and  Efficiency,  "Quality  Standards 
for  Inspection  and  Evaluation."  Those  standards  require  that  we  plan  and  perform  the 
assessment  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  assessment  objectives.  We  believe  that  the 
evidence  obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on 
our  assessment  objectives. 

To  review  the  quality  management  system,  our  assessments  focused  on  the  following: 

•  Applicable  Statutory/regulatory  requirements 

•  Contractual  quality  management  system  (AS9100] 

•  Contractual  quantity  clauses 

•  Internal  quality  assurance  processes  and  procedures 

•  Software  quality  assurance  process 

•  Aviation  CSI 

Use  of  Technical  Assistance 

We  used  assistance  from  quality  assurance  engineers  and  quality  assurance  specialists 
with  a  background  in  defense  and  aerospace  systems.  We  established  teams  of  subject 
matter  experts  who  assessed  to  the  AS9100C  Quality  Management  System  standard. 
The  subject  matter  expert  teams  consisted  of  14  to  18  quality  assurance  engineers, 
trained  and  certified  in  AS9100,  who  had  an  average  of  15  years  of  quality  assurance, 
audit  experience.  Additionally,  our  teams  included  subject  matter  experts  in  military 
aviation  CSI. 
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Prior  Coverage 

During  the  last  5  years,  the  Government  Accountability  Office  (GAO]  and  the 
Department  of  Defense  Inspector  General  (DoD  IG]  issued  26  reports  discussing 
F-35  Joint  Strike  Fighter.  Unrestricted  GAO  reports  can  be  accessed  over  the 
Internet  at  http://www.gao.gov.  Unrestricted  DoD  IG  reports  can  be  accessed  at 

http :  /  /  www.dodig.mil/pubs/index.cfm. 

GAO 

Report  No.  GAO-13-690T,  "F-35  Joint  Strike  Fighter:  Restructuring  Has  Improved  the 
Program,  but  Affordability  Challenges  and  Other  Risks  Remain,”  June  19,  2013 

Report  No.  GAO-13-294SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,”  March  28,  2013 

Report  No.  GAO-13-309,  "F-35  Joint  Strike  Fighter:  Current  Outlook  Is  Improved,  but 
Long-Term  Affordability  Is  a  Major  Concern,"  March  11,  2013 

Report  No.  GAO-12-437,  "Joint  Strike  Fighter:  DOD  Actions  Needed  to  Further  Enhance 
Restructuring  and  Address  Affordability  Risks,"  June  14,  2012 

Report  No.  GAO-12-400SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,”  March  29,  2012 

Report  No.  GAO-12-525T,  "Joint  Strike  Fighter:  Restructuring  Added  Resources  and 
Reduced  Risk,  but  Concurrency  Is  Still  a  Major  Concern,”  March  20,  2012 

Report  No.  GAO-11-903R,  "Joint  Strike  Fighter:  Implications  of  Program  Restructuring 
and  Other  Recent  Developments  on  Key  Aspects  of  DOD's  Prior  Alternate  Engine 
Analyses,"  September  14,  2011 

Report  No.  GA0-11-677T,  "Joint  Strike  Fighter:  Restructuring  Places  Program  on  Firmer 
Footing,  but  Progress  Is  Still  Lagging,"  May  19,  2011 

Report  No.  GAO-11-325,  "Joint  Strike  Fighter:  Restructuring  Places  Program  on  Firmer 
Footing,  but  Progress  Still  Lags,"  April  7,  2011 
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Report  No.  GAO-11-450T,  "Joint  Strike  Fighter:  Restructuring  Should  Improve  Outcomes, 
but  Progress  Is  Still  Lagging  Overall,"  March  15,  2011 

Report  No.  GAO-11-323R,  "Tactical  Aircraft:  Air  Force  Fighter  Force  Structure  Reports 
Generally  Addressed  Congressional  Mandates,  but  Reflected  Dated  Plans  and  Guidance, 
and  Limited  Analyses,”  February  24,  2011 

Report  No.  GAO-11-171R,  "Defense  Management:  DOD  Needs  to  Monitor  and  Assess 
Corrective  Actions  Resulting  from  Its  Corrosion  Study  of  the  F-35  Joint  Strike  Fighter," 
December  16,  2010 

Report  No.  GAO  10-1020R,  "Joint  Strike  Fighter:  Assessment  of  DOD's  Funding  Projection 
for  the  F136  Alternate  Engine,"  September  15,  2010 

Report  No.  GAO-10-789,  "Tactical  Aircraft:  DOD's  Ability  to  Meet  Future  Requirements 
is  Uncertain,  with  Key  Analyses  Needed  to  Inform  Upcoming  Investment  Decisions," 
July  29,  2010 

Report  No.  GAO-10-388SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,"  March  30,  2010 

Report  No.  GAO-10-478T,  "Joint  Strike  Fighter:  Significant  Challenges  and  Decisions 
Ahead,"  March  24,2010 

Report  No.  GAO-10-382,  "Joint  Strike  Fighter:  Additional  Costs  and  Delays  Risk  Not 
Meeting  Warfighter  Requirements  on  Time,"  March  19,  2010 

Report  No.  GAO-10-520T,  "Joint  Strike  Fighter:  Significant  Challenges  Remain  as  DOD 
Restructures  Program,"  March  11,  2010 

Report  No.  GAO-09-71  IT,  "Joint  Strike  Fighter:  Strong  Risk  Management  Essential  as 
Program  Enters  Most  Challenging  Phase,"  May  20,  2009 

Report  No.  GAO-09-326SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,"  March  30,  2009 

Report  No.  GAO-09-303,  "Joint  Strike  Fighter:  Accelerating  Procurement  before 
Completing  Development  Increases  the  Government's  Financial  Risk,"  March  12,  2009 
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Report  No.  GAO-08-782T,  "Defense  Acquisitions:  Better  Weapon  Program  Outcomes 
Require  Discipline,  Accountability,  and  Fundamental  Changes  in  the  Acquisition 
Environment,”  June  3,  2008 

Report  No.  GAO-08-467SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,”  March  31,  2008 

Report  No.  GAO-08-569T,  "Joint  Strike  Fighter:  Impact  of  Recent  Decisions  on  Program 
Risks,"  March  11,  2008 

Report  No.  GAO-08-388,  "Joint  Strike  Fighter:  Recent  Decisions  by  DOD  Add  to  Program 
Risks,"  March  11,  2008 

DoD  IG 

Report  No.  DODIG-2013-031,  "Audit  of  the  F-35  Lightning  II  Autonomic  Logistics 
Information  System  (ALISJ,"  December  10,  2012 
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Appendix  C 

Notices  of  Concern  and  JPO  Responses 

NOC  Lockheed  Martin 


INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 

4800  MARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA  22350-1500 


APR  2  5  2012 


MEMORANDUM  FOR  PROGRAM  EXECUTIVE  OFFICE  JOINT  STRIKE  FIGHTER 

SUBJECT:  Notice  of  Concern-Quality  Assurance  Assessment  of  the  F-35  Program 
(Project  No.  D2012-DT0TAD-0003) 

We  are  issuing  this  Notice  of  Concern  (NOC)  to  inform  you  that  the  Department  of 
Defense,  Inspector  General  (DoD  IG)  team  identified  issues  that  require  your  attention.  During 
the  Quality  Assurance  assessment1  of  the  F-35  program  at  Lockheed  Martin  Aeronautics  (LMA), 
Fort  Worth,  Texas,  from  March  26,  2012  to  April  6,  2012,  the  team  identified  70  findings,  29  of 
which  we  consider  to  be  major  (systemic)  and  41  to  be  minor. 

We  sorted  the  findings  into  six  broad  categories:  Document  Control,  Risk  Management, 
Process  Discipline,  Process  Proofing,  Foreign  Object  Debris  (FOD),  and  Critical  Safety  Items 
(CSI).  Our  assessment  of  each  of  these  areas  is  as  follows: 

•  Document  and  configuration  control  is  weak  and  needs  management  attention. 

•  Risk  management  needs  to  be  handled  upfront  programmatically  and  technically. 

•  Process  discipline  is  lacking  in  many  areas  reviewed  by  the  team. 

•  Production  processes  have  not  been  thoroughly  proofed. 

•  FOD  discipline  is  lacking  and  requires  major  cultural  changes. 

•  CSI  Development  needs  immediate  and  complete  implementation. 

On  average,  at  final  assembly  each  aircraft  has  200+  corrective  actions  requiring  rework 
or  repair.  The  DoD  IG  team’s  overall  conclusion  is  that  LMA’s,  Fort  Worth,  Texas  quality 
management  system  and  the  integrity  of  the  F-35  product  are  jeopardized  by  a  lack  of  attention 
to  detail,  inadequate  process  discipline,  and  a  “we  will  catch  it  later”  culture.  We  believe  the 
quality  assurance  culture  at  LMA,  Fort  Worth,  Texas  must  improve  and  that  robust  technical 
oversight  by  the  government  is  required  to  ensure  program  performance  and  mission  success. 

Our  findings  are  attached  to  this  memorandum  for  your  review  and  comment. 

DoD  Directive  7650.3  requires  that  recommendations  be  resolved  promptly.  Please  provide 
comments  that  state  whether  you  agree  or  disagree  with  the  findings  and  recommendations.  If 
you  agree  with  our  recommendations,  describe  what  actions  you  have  taken  or  plan  to  take  to 
accomplish  the  recommendations  and  include  the  completion  dates  of  your  actions.  If  you 
disagree  with  the  recommendations  or  any  part  of  them,  please  give  specific  reasons  why  you 
disagree  and  propose  alternative  action  if  appropriate.  Once  we  complete  our  five  assessments, 


1  The  assessment  was  conducted  in  accordance  with  the  Council  of  the  Inspector  General  on 
Integrity  and  Efficiency  (CIGIE)  Quality  Standards  for  Inspection  and  Evaluation  and  guidance 
in  AS9101,  Quality  Management  Systems  Audit  Requirements  for  Aviation,  Space,  and  Defense 
Organizations. 
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NOC  Lockheed  Martin  (cont'd) 


we  will  provide  a  report  on  our  findings,  along  with  any  additional  matters  that  might  come  to 
our  attention. 

We  should  receive  your  comments  by  May  25.  2012.  We  normally  include  copies  of  the 
comments  in  the  report.  If  you  consider  any  matters  to  be  exempt  from  public  release,  you 
should  mark  lliem  clearly  for  Inspector  General  consideration. 

^^^Iimossibl^end  a  portable  document  format  (.pdl)  tile  containing  your  comments  to 

!.f  •  (  opies  of  soui  comment'  must  line  the  actual  Munaluie  of  the 

aumonzms^tncianor  your  organization.  We  are  unable  to  accept  the  /Signed/  symbol  in  place 
of  the  actual  signature.  If  you  arrange  to  send  classified  comments  electronically,  you  must  send 
them  over  the  SECRET  Internet  Protocol  Router  Network  (SIPRNET). 


Randolph  R.  Stone,  SES 
Deputy  Inspector  General 
Policy  and  Oversight 


Attachments: 
As  staled 
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JPO  Response  to  NOC  Lockheed  Martin 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


From:  Program  Executive  Officer,  F-35  Lightning  II 

To:  Deputy  Inspector  General,  Policy  and  Oversight  MAY  2 3  2( 

Department  of  Defense  Inspector  General 

Sub j !  RESPONSE,  NOTICE  OF  CONCERN  -  QUALITY  ASSURANCE 
ASSESSMENT  OF  THE  F-35  PROGRAM 

1.  I  received  and  analyzed  your  25  April  2012  Notice  of 
Concern-Quality  Assurance  Assessment  of  the  F-35  Program 
(Project  No.  D2012-DTOTAD-0003 )  regarding  your  visit  to  Lockheed 
Martin  Aeronautics  (LMA) ,  Fort  Worth,  Texas  from  26  March  to  6 
April  2012.  I  appreciate  your  team's  efforts  to  assess 
conformity  to  F-35  quality  management  systems,  contractual 
quality  clauses,  and  internal  processes  and  procedures,  to 
include  Aviation  Critical  Safety  Items.  I  view  your  assessment 
as  a  valuable  tool  to  independently  evaluate  the  areas  for 
improvement  within  the  F-35  program,  with  the  ultimate  goal  of 
producing  a  quality,  timely,  and  cost-effective  weapon  system 
for  the  United  States  and  our  allies. 

2.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the 
Defense  Contract  Management  Agency  (dcma)  and  LMA,  reviewed  each 
of  the  assessment's  findings  and  recommendations.  I  accept  all 
of  the  findings  issued  relative  to  LMA  as  valid.  I  agree  that 
all  of  the  recommendations  issued  relative  to  LMA  should  be 
assessed  and  implemented  to  the  maximum  practicable  extent. 
Additionally,  the  F-35  JPO  reviewed  the  findings  and 
recommendations  issued  relative  to  the  F-35  JPO,  and  I  also 
accept  all  of  these  findings  as  valid.  I  accept  all  of  the 
recommendations  issued  relative  to  the  F-35  JPO.  A  detailed 
discussion  of  our  path  forward  is  contained  at  Appendix  A  and  is 
marked  for  Inspector  General  Consideration. 

3.  We  will  continue  to  work  with  you  and  your  team  during  the 
quality  assurance  assessments  of  selected  LMA  subcontractors. 


Vice  Admiral,  U.S.  Navy 


Appendix  A 
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JPO  Response  to  NOC  Lockheed  Martin  (cont'd) 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


Copy  to: 

OUSD (AT&L) 

Director,  Engineering 

JSF  Weapon  System  Program  Manager 

Director,  Production 

Director,  Program  Integration 
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JPO  Response  to  NOC  Lockheed  Martin  (cont'd) 


FOR  INSPECTOR 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12 rH  Street  South,  Suite  600 
Arlington.  Virginia  22202-5402 

GENERAL  CONSIDERATION 


Appendix  A: 

1.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the 
Defense  Contract  Management  Agency  (DCMA)  and  LMA,  reviewed  each 
of  the  assessment's  findings  and  recommendations.  I  accept  all 
of  the  findings  issued  relative  to  LMA  as  valid.  T  agree  that 
all  of  the  recommendations  issued  relative  to  LMA  should  be 
assessed  and  implemented  to  the  maximum  practicable  extent. 
Additionally,  the  F-35  JPO  reviewed  the  findings  and 
recommendations  issued  relative  to  the  F-35  JPO,  and  I  also 
accept  all  of  these  findings  as  valid.  I  accept  all  of  the 
recommendations  issued  relative  to  the  F-35  JPO  and  will 
implement  them  as  discussed  below.  Although  we  may  discover, 
during  our  root  cause  analysis  and  development  of  corrective 
action  plans,  that  a  particular  aspect  of  a  finding  may  be 
already  addressed  and  fixed  to  satisfy  requirements,  I  believe 
that  every  finding  has  a  deficiency  that  needs  to  be  further 
addressed  by  our  program  office.  Further,  I  intend  to  ensure 
that  any  corrections  and  improvements  do  not  stay  restricted  to 
one  location  but  are  distributed  and  implemented  throughout  the 
JSF  enterprise. 


2.  Regarding  the  findings  issued  relative  to  LMA,  the  F-35 
JPO's  first  step  was  to  establish  a  joint  working  group  among 
the  F-35  JPO,  LMA,  and  DCMA  to  ensure  that  all  of  the  findings 
and  recommendations  are  understood  and  to  start  developing 
corrective  action  plans.  Points  of  Contact  were  established  by 
9  May  2012  for  every  finding  within  each  of  the  three 
organizations,  and  discussions  have  already  commenced.  The  F-35 
JPO  then  notified  DCMA  of  our  intent  to  accept  all  of  the 
findings  (14  May  2012)  .  DCMA  is  issuing  Level  2  Corrective 
Action  Requests  (CARS)  for  each  finding  (on-going,  ECD  31  May 
2012) .  The  DCMA  CAR  process  provides  the  Government  a 
systematic  approach  to  ensure  the  Contractor  takes  corrective 
action  in  a  timely  manner.  The  F-35  JPO,  in  concert  with  the 
responsible  DCMA  organization,  will  implement  monthly 
adjudication  meetings  with  each  location  included  in  the 
assessment  to  ensure  appropriate  progress  is  being  accomplished 
in  the  corrective  action  plan  for  each  finding.  The  first 
meeting  with  LMA  will  be  held  in  July  2012. 

3.  Regarding  the  findings  issued  relative  to  the  F-35  JPO,  we 
are  pursuing  the  following  process. 
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JPO  Response  to  NOC  Lockheed  Martin  (cont'd) 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 

a.  Of  the  seven  findings  issued  relative  to  the  F-35  JPO, 
two  address  concerns  regarding  documentation  of  meetings  for  the 
Program  Management  Advisory  Board  and  Test  Readiness  Reviews. 

In  accordance  with  your  recommendations,  this  will  be  addressed 
by  31  August  2012. 

b.  Four  findings  deal  with  the  program  implementation  of 
Critical  Safety  Item  (CSI)  management.  Your  recommendations 
direct  the  F-35  JPO  to  implement  a  fully  realized  CSI  plan, 
which  we  fully  support.  An  F-35  CSI-Specific  Implementation 
Policy  has  been  developed.  That  document  is  being  updated  to 
communicate  the  additional  CSI  requirements  and  guidance  in 
order  to  address  the  DoDIG's  findings.  The  updated  policy  will 
form  the  basis  of  the  JPO  plans  to  put  the  Prime  Contractor  on 
contract  for  Phase  I  (Prime  Contractor  CSI)  and  Phase  II 
(Supplier  CSI)  CSI  Non-recurring  engineering  (NRE)effort,  as 
well  as  the  CSI  recurring  efforts.  We  plan  to  work  with  our 
Prime  Contractor  in  the  next  several  weeks  to  develop  a  CSI 
roadmap/ schedule  for  the  CSI  program.  The  F-35  JPO  is  working  to 
update  and  authorize  the  Phase  I  CSI  nre  request  for  proposal. 

c.  The  F-35  JPO  accepts  the  final  finding  , 4LM005,  which 
identifies  the  following  issues:  "a)  JPO  does  not  have  a 
(Diminishing  Manufacturing  Sources  and  Material  Shortages]  DMSMS 
Program  Management  Plan  as  required  by  SECNAVINST  5000. 2E  (and 
previous  revisions)  or  DoD  4140. 1R.  b)  DMSMS  management 
activities  have  not  been  funded  in  the  Dow  Rate  Initial 
Production  (LRIP)  4  and  5  contracts.  Therefore,  a  proactive 
DMSMS  management  process  is  not  being  implemented  by  LM  or  its 
suppliers  on  the  F-35."  We  fully  accept  the  first  part  of  the 
finding  regarding  the  lack  of  a  DMSMS  Program  Management  Plan, 
but  it  is  important  to  point  out  that  we  do  have  some  DMSMS 
activities  on  contract.  Prior  to  8  July  2011,  DMS  non¬ 
recurring/redesign  and  management  activities  were  performed 
under  the  SDD  contract.  After  that  date,  DMS  management 
activities  are  being  performed  under  the  sustainment  portion  of 
the  LRIP  5  contract.  Beginning  with  the  def initization  of  the 
LRIP  5  production  Undefinitized  Contract  Action,  which  is 
currently  being  negotiated,  all  future  DMS  management  and  non¬ 
recurring/redesign  activities  will  be  included  as  part  of  the 
production  contract.  The  JPO  fully  supports  your  recommendation 
to  implement  a  robust  DMSMS  program  and  flow  that  down  to  LMA . 
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NOC  Northrop  Grumman 


INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 
480D  MARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA  22360  1600 

JUN  1  3  2012 


MEMORANDUM  FOR  PROGRAM  EXECUT  IVE  OFFICE  JOINT  STRIKE  FIGHTER 

SUBJECT:  Notice  of  Concern-Quality  Assurance  Assessment  of  the  F-35  Program 
(Project  No.  D20 1 2-DTOT  AD-0003) 

We  are  issuing  Ihis  Notice  of  Concern  to  inform  you  lhal  the  Department  of  Defense, 
Inspector  General  (DoD  IG)  team  identified  issues  that  require  your  attention.  During  the 
Quality  Assurance  Assessment1  of  the  F-3S  program  at  Northrop  Grumman  Corporation  (NGC). 
El  Segundo  and  Palmdale,  California,  from  May  7, 2012  to  May  18,  2012,  the  team  identified 
66  findings. 

Wc  arranged  the  findings  into  five  broad  categories:  Document  Control,  Requirements 
Flow-Down,  Process  Proofing,  Process  Discipline,  and  Tool  and  Equipment  Control.  Our 
assessment  of  each  of  these  areas  is  as  follows: 

•  Document  Control-Plans,  procedures,  and  records  were  not  always  maintained 

•  Requirements  Flow-Down-Contractual  and/or  Federal  Acquisition  Regulation 
requirements  were  not  always  correctly  (lowed  down  and/or  implemented 

•  Process  Proofing  Production  processes  and  work  instructions  were  not  well-defined 
and/or  incomplete 

•  Process  Discipline-Failure  to  follow  established  procedures  was  prevalent  throughout 
most  quality  management  processes 

•  Tool  and  Equipment  Control  Tools  and/or  equipment  were  not  properly  identified, 
proofed,  and  controlled 

The  DoD  IG  team’s  overall  conclusion  is  that  NGC’s  quality  management  system  is  no! 
always  implemented  or  sufficiently  defined;  however,  management  is  actively  engaged  in  the 
day-to-day  production  activities  and  proactively  working  issues  as  they  arise.  Their  general  atlilude 
was  lhal  the  issues  noted  by  the  DoD  IG  assessment  team  were  opportunities  to  improve  their 
product. 


1  We  conducted  the  assessment  in  accordance  with  the  Council  of  the  Inspector  General  on 
Integrity  and  Efficiency  Quality  Standards  for  Inspection  and  Evaluation  and  guidance  in 
AS9101,  Quality  Management  Systems  Audit  Requirements  for  Aviation,  Space,  and  Defense 
Organizations. 
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NOC  Northrop  Grumman  (cont'd) 


Our  findings  are  attached  to  this  memorandum  for  your  review  and  comment. 

DoD  Directive  7650.3  requires  that  recommendations  be  resolved  promptly.  Please  provide 
comments  that  state  whether  you  agree  or  disagree  with  the  findings  and  recommendations.  If 
you  agree  with  our  recommendations,  describe  what  actions  you  have  taken  or  plan  to  take  to 
accomplish  the  recommendations  and  include  the  completion  dates  of  your  actions.  If  you 
disagree  with  the  recommendations  or  any  part  of  them,  please  give  specific  reasons  why  you 
disagree  and  propose  alternative  action  if  appropriate.  Once  we  complete  our  five  assessments, 
we  will  provide  a  report  on  our  findings,  along  with  any  additional  matters  that  might  come  to 
our  attention. 

We  should  receive  your  comments  by  July  1 3, 2012.  Wc  normally  include  copies  of  the 
comments  in  the  report.  If  you  consider  any  matters  to  be  exempt  from  public  release,  you 
should  mark  them  clearly  for  Inspector  General  consideration. 

Ifpossible,  send  a  portable  document  format  (.pdf)  file  containing  your  comments  to 


.  Copies  of  your  comments  must  have  the  actual  signature  of  the 


authorizing  official  for  your  organization.  Wc  are  unable  to  accept  the  /Signed/  symbol  in  place 
of  the  actual  signature.  If  you  arrange  to  send  classified  comments  electronically,  you  must  send 
them  over  the  SECRET  Internet  Protocol  Router  Network  (SIPRNET). 

We  appreciate  the  courtesies  extended  to  the  staff.  Please  direct  questions  to  


Randolph  R.  Stone 
Deputy  Inspector  General 


Policy  and  Oversight 


Attachments: 
As  stated 


2 


84  |  DODIG-2013-140 


Appendixes 


JPO  Response  to  NOC  Northrop  Grumman 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 

From:  Program  Executive  Officer,  F-35  Lightning  II 

To:  Deputy  Inspector  General,  Policy  and  Oversight 

Department  of  Defense  Inspector  General 

Subj :  RESPONSE,  NOTICE  OF  CONCERN  -  QUALITY  ASSURANCE 

ASSESSMENT  OF  THE  F-35  PROGRAM 

1.  I  received  and  analyzed  your  13  Jun  2012  Notice  of  Concern- 
Quality  Assurance  Assessment  of  the  F-35  Program  (Project  No. 
D2012-DTDTAD-0003 )  regarding  your  visit  to  Northrop  Grumman 
Corporation  (NGC ) ,  El  Segundo  and  Palmdale,  California,  '/-lb  May 
2012.  I  appreciate  your  team's  efforts  to  assess  conformity  to 
F-35  quality  management  systems,  contractual  quality  clauses, 
and  internal  processes  and  procedures,  to  include  Aviation 
Critical  Safety  Items.  I  view  your  assessment  as  a  valuable 
tool  to  independently  evaluate  the  areas  for  improvement  within 
the  F-35  Program,  with  the  ultimate  goal  of  producing  a  quality, 
timely,  and  cost-effective  weapon  system  for  the  united  States 
and  our  allies. 


2.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the 
Defense  Contract  Management  Agency  (DCMA) ,  NGC,  and  the  prime 
contractor,  Lockheed  Martin  Aeronautics  (LMA) ,  reviewed  each  of 
the  assessment's  findings  and  recommendations.  I  accept  all  of 
the  findings  issued  relative  to  NGC  and  LMA  as  valid.  I  agree 
that  all  of  the  recommendations  issued  relative  to  NGC  and  LMA 
should  be  assessed  and  implemented  to  the  maximum  practicable 
extent.  A  detailed  discussion  of  our  path  forward  is  contained 
at  Appendix  A  and  is  marked  for  Inspector  General  Consideration 

3.  We  will  continue  to  work  with  you  and  your  team  during  the 
quality  assurance  assessments  of  the  remaining  selected  LMA 
subcontractors . 

DAVID  J./VENLET 

Vice  Admiral,  U.S.  Navy 


Appendix  A 

Copy  to: 
OUSD ( AT&L I 


2012 
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JPO  Response  to  NOC  Northrop  Grumman  (cont'd) 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 

FOR  INSPECTOR  GENERAL  CONSIDERATION 

Appendix  A: 

1.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the 
Defense  Contract  Management  Agency  (DCMA) ,  NGC,  and  the  prime 
contractor,  Lockheed  Martin  Aeronautics  (LMA) ,  reviewed  each  of 
the  assessment’s  findings  and  recommendations.  I  accept  all  of 
the  findings  issued  relative  to  NGC  and  LMA  as  valid.  I  agree 
that  all  of  the  recommendations  issued  relative  to  NGC  and  LMA 
should  be  assessed  and  implemented  to  the  maximum  practicable 
extent.  Although  we  may  discover,  during  our  root  cause 
analysis  and  development  of  corrective  action  plans,  that  a 
particular  aspect  of  a  finding  may  be  already  addressed  and 
fixed  to  satisfy  requirements,  I  believe  that  every  finding  has 
a  deficiency  that  needs  to  be  further  addressed  by  our  program 
office.  Further.  I  intend  to  ensure  that  any  corrections  and 
improvements  do  not  stay  restricted  to  one  location  but  are 
distributed  and  implemented  throughout  the  JSF  enterprise. 

2.  Regarding  the  findings  issued  relative  to  NGC,  the  F-35  JPO 
established  a  joint  working  group  among  the  F-35  JPO,  NGC,  LMA, 
and  DCMA  to  ensure  that  all  of  the  findings  and  recommendations 
are  understood  and  to  start  developing  corrective  action  plans. 
Points  of  Contact  within  each  of  the  four  government 
organizations  will  be  established  for  every  finding  by  18  July, 
but  discussions  have  already  commenced.  The  F-35  JPO  has  begun 
working  with  DCMA  at  both  NGC  sites  to  understand  and  properly 
characterize  each  finding  with  an  eye  towards  drafting 
Corrective  Action  Requests  (CARs)  for  appropriate  findings  (on¬ 
going,  ecd  18  July  2012) .  The  DCMA  CAR  process  provides  the 
Government  a  systematic  approach  to  ensure  that  corrective 
actions  are  taken  in  a  timely  manner.  The  F-35  JPO,  in  concert 
with  the  responsible  DCMA  organization  and  LMA,  will  implement 
monthly  adjudication  meetings  with  each  location  included  in  the 
assessment  to  ensure  appropriate  progress  is  being  accomplished 
in  the  corrective  action  plan  for  each  finding.  The  first 
meeting  with  NGC.  LMA,  and  DCMA  will  be  held  in  August  2012. 

3.  Findings  issued  relative  to  LMA  have  been  rolled  into  the 
tracking  process  initiated  during  the  initial  visit  to  LMA  and 
will  be  included  in  the  LMA  monthly  tracking  meetings. 
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NOC  L-3  Communications 


INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 
4800  MARK  CENTER  DRIVE 
ALEXANDRIA.  VIRGINIA  22350-1500 


JUL  |  |  2012 


MEMORANDUM  FOR  PROGRAM  EXECUTIVE  OFFICE  JOINT  S I KIKE  FIGHTER 

SUBJECT:  Notice  of  Concern-Quality  Assurance  Assessment  of  the  F-35  Program 
(Project  No.  02012-010  I  AI)-!I003) 

We  are  issuing  this  Notice  of  Concern  to  inform  yon  that  the  Department  of  Defense, 
Inspector  General  (DoD  IG)  team  identified  issues  that  require  your  attention.  During  the 
Quality  Assurance  Assessment  of  the  F-35  program  at  L-3  Communications,  Alpharetta, 
Georgia,  from  June  1 8, 201 2  to  June  22, 2012,  the  team  identified  56  findings. 

We  sorted  the  findings  into  five  broad  categories:  Process  Discipline,  Document  Control, 
Requirements  Flow-Down,  fool  and  Equipment  Control,  and  Process  Proofing.  Our  assessment 
of  each  of  these  areas  is  as  follows 

•  Process  Discipline-Failure  to  follow  established  procedures  was  prevalent  throughout 

most  quality  management  processes  reviewed 

•  Document  Coutiol  Plans,  procedures,  and  records  were  not  always  maintained 

•  Requirements  Flow- Down  Contractual  and/or  f  ederal  Acquisition  Regulation 

requirements  were  not  always  correctly  flowed  down  and/or  implemented 

■  fool  and  Equipment  Control  Tools  and/or  equipment  were  not  properly  controlled,  and 

the  calibration  system  had  serious  deficiencies 

•  Process  Proofing  Production  processes  and  work  instructions  were  not  well-defined 

and/or  incomplete 

The  DoD  IG  team's  overall  conclusion  is  that  I.-3’s  quality  management  system  is  not 
always  implemented  or  sufficiently  defined.  Approximately  half  (43  percent)  of  the  findings 
were  in  process  discipline,  indicating  a  lack  of  attention  to  detail,  hr  addition,  the  calibration 
system  laid  serious  deficiencies  that  warrant  a  complete  review,  and  first  article  inspections  were 
not  being  performed  as  required. 

Our  findings  are  attached  to  this  memorandum  for  your  review  and  comment, 

DoD  Directive  7650.3  requires  that  recommendations  be  resolved  promptly.  Please  provide 
comments  that  state  whether  you  agree  or  disagree  with  the  findings  and  recommendations.  If 
you  agree  with  our  recommendations,  describe  what  actions  you  have  taken  or  plan  to  take  to 
accotnplish  the  recommendations  and  include  the  completion  dates  of  your  actions.  If  you 
disagree  with  the  recommendations  or  any  purl  of  them,  please  give  specific  reasons  why  you 

1  We  conducted  the  assessment  in  accordance  with  the  Council  of  the  Inspector  General  on 
Integrity  and  Efficiency  Quality  Standards  for  Inspection  and  Evaluation  and  guidance  in 
AS91 0 1 ,  Quality  Management  Systems  Audit  Requirements  for  Aviation,  Space,  and  Defense 
Organizations. 
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NOC  L-3  Communications  (cont'd) 


disagree  and  propose  alternative  action  if  appropriate.  Once  we  complete  our  assessments,  we 
will  provide  a  report  on  our  findings,  along  with  any  additional  mallets  that  might  come  to  out 
attention, 

Wc  should  receive  your  comments  by  August  1 3, 2012.  Wc  normally  include  copies  of 
the  comments  in  the  report.  If  you  consider  any  matters  to  be  exempt  from  public  release,  you 
should  mark  them  clearly  lor  Inspector  General  consideration. 

If  possible,  send  a  portable  document  format  (pdf)  file  containing  your  comments  to 


|.  Copies  of  your  comments  must  have  the  actual  signature  of  the 


authorizing  official  for  your  organization.  We  are  unable  to  accept  the  /Signed/  symbol  in  place 
of  the  actual  signature.  If  you  arrange  to  send  classified  comments  electronically,  you  must  send 
them  over  the  SECRET  Internet  Protocol  Router  Network  (S1PRNET). 

We  appreciate  the  courtesies  extended  to  the  staff.  Please  direct  questions  to  


Policy  and  Oversight 


Attachment: 
As  stated 


2 
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JPO  Response  to  NOC  L-3  Communications 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


Prom:  Program  Executive  Officer,  F-35  Lightning  II 

To:  Deputy  Inspector  General,  Policy  and  Oversight 

Department  ol  Detense  Inspector  General 

Sub j :  RESPONSE,  NOTICE  OF  CONCERN  QUALITY  ASSURANCE 
ASSESSMENT  OF  THE  F-35  PROGRAM 

1.  1  received  and  analyzed  your  11  Jul  2012  Notice  of  Concern- 
Quality  Assurance  Assessment  of  the  F-35  Program  (Project  No. 
D2012-DT0TAD-0003 )  regarding  your  visit  to  L-3  Communications, 
Alpharetta,  Georgia,  18-22  June  2012.  I  appreciate  your  team's 
efforts  to  assess  conformity  to  F-35  quality  management  systems, 
contractual  quality  clauses,  and  internal  processes  and 
procedures.  I  view  your  assessment  as  a  valuable  tool  to 
independently  evaluate  the  areas  for  improvement  within  the  F-35 
Program,  with  the  ultimate  goal  of  producing  a  quality,  timely, 
and  cost-effective  weapon  system  for  the  united  States  and  our 
allies . 

2.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the 
Defense  Contract  Management  Agency  (DCMA) ,  L-3,  and  the  prime 
contractor,  Lockheed  Martin  Aeronautics  (LMA) ,  reviewed  each  of 
the  assessment's  findings  and  recommendations.  I  accept  all  of 
the  findings  issued  relative  to  L-3  and  LMA  as  valid.  I  agree 
that  all  of  the  recommendations  issued  relative  to  L-3  and  LMA 
should  be  assessed  and  implemented  to  the  maximum  practicable 
extent.  The  JPO  acknowledges  the  findings  issued  against  DCMA 
Fort  Worth  and  DCMA  Atlanta,  and  will  collaborate  with  DCMA  to 
ensure  recommendations  are  assessed  and  implemented  to  the 
maximum  practicable  extent.  A  detailed  discussion  of  our  path 
forward  is  contained  at  Appendix  A  and  is  marked  for  Inspector 
General  Consideration. 

3.  We  will  continue  to  work  with  you  and  your  team  during  the 
quality  assurance  assessments  of  the  remaining  selected  LMA 
subcontractors . 


DAVID  J.  tfENLET 

Vice  Admiral,  U.S.  Navy 
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JPO  Response  to  NOC  L-3  Communications  (cont'd) 


JOINT  STRIKE  FIGHTER  PROGR  AM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


FOR  INSPECTOR  GENERAL  CONSIDERATION 
Appendix  A: 

1.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the 
Defense  Contract  Management  Agency  (DCMA) ,  L-3,  and  the  prime 
contractor,  Lockheed  Martin  Aeronautics  (LMA) ,  reviewed  each  of 
the  assessment's  findings  and  recommendations.  T  accept  all  of 
the  findings  issued  relative  to  L-3  and  LMA  as  valid.  I  agree 
that  all  of  the  recommendations  issued  relative  to  L-3  and  LMA 
should  be  assessed  and  implemented  to  the  maximum  practicable 
extent.  Although  we  may  discover,  during  our  root  cause 
analysis  and  development  of  corrective  action  plans,  that  a 
particular  aspect  of  a  finding  may  be  already  addressed  and 
fixed  to  satisfy  requirements,  I  believe  that  every  finding  has 
a  deficiency  that  needs  to  be  further  addressed  by  our  program 
office.  Further,  I  intend  to  ensure  that  any  corrections  and 
improvements  do  not  stay  restricted  to  one  location  but  are 
distributed  and  implemented  throughout  the  JSF  enterprise. 

2.  Regarding  the  findings  issued  relative  to  L-3,  the  F-35  JPO 
established  a  -joint  working  group  among  the  F-35  JPO,  L-3,  LMA. 
and  DCMA  to  ensure  that  all  of  the  findings  and  recommendations 
are  understood  and  to  start  developing  corrective  action  plans . 
Points  of  Contact  within  each  of  the  four  organizations  will  be 
established  for  every  finding  by  10  August  2012,  but  discussions 
have  already  commenced.  The  F-35  JPO  has  begun  working  with 
DCMA  at  L-3  to  understand  and  properly  characterize  each  finding 
with  an  eye  towards  drafting  Corrective  Action  Requests  (CARs) 
for  appropriate  findings  (on-going,  ECD  31  August  2012) .  The 
DCMA  CAR  process  provides  the  Government  a  systematic  approach 
to  ensure  that  corrective  actions  are  taken  in  a  timely  manner. 
The  F-35  JPO,  in  concert  with  the  responsible  DCMA  organization 
and  LMA,  will  implement  monthly  adjudication  meetings  with  each 
location  included  in  the  assessment  to  ensure  appropriate 
progress  is  being  accomplished  in  the  corrective  action  plan  for 
each  finding.  The  first  meeting  with  L-3,  LMA,  and  DCMA  will  be 
held  in  September  2012 . 

3.  Findings  issued  relative  to  LMA  have  been  rolled  into  the 
tracking  process  initiated  during  the  initial  visit  to  LMA  and 
will  be  included  in  the  LMA  monthly  tracking  meetings. 

4.  Findings  issued  relative  to  DCMA  Fort  Worth  and  DCMA  Atlanta 
are  being  actively  addressed.  DCMA  Atlanta  has  ensured  proper 
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JPO  Response  to  NOC  L-3  Communications  (cont'd) 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


procedures  are  in  place  and  personnel  properly  trained  to 
conduct  the  required  inspections.  They  have  developed  a  revised 
Quality  Assurance  Surveillance  plan  for  L-3  Display  Systems  and 
are  developing  an  internal  procedure  for  “Delegation  Review  and 
Receipt,"  which  will  be  completed  by  3i  August  2012. 


DMCA  Fort  Worth  has  reviewed  their  Letters  of  Delegation  to 
ensure  all  information  is  correctly  reflected,  which  is  expected 
to  be  complete  by  31  August  2012.  DCMA  Fort  Worth  is  also 
modifying  their  F-35  Suppliers  Information  Delegation  to  include 
a  Monthly  Reporting  Requirements  column,  and  will  follow  up 
within  one  business  day  if  the  deadline  for  reporting  is  missed. 
Finally,  DCMA  is  implementing  a  new  procedure  to  extract /compile 
and  forward  to  the  cognizant  DCMA  office  performance  data 
relating  to  those  assigned  suppliers  whose  performance  is  below 
LM's  Supplier  Quality  Rating  or  that  have  received  related 
product  deficiency  reports.  They  are  updating  their 
Surveillance  strategy  and  desk  top  instructions  to  reflect  this 
change,  which  will  be  completed  by  30  September  2012. 
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NOC  Honeywell  -  Phoenix ,  AZ 


DEPARTMENT  OF  DEFENSE 
4800  MARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA  22350-1500 


INSPECTOR  GENERAL 


JUL  26  2012 


MEMORANDUM  FOR  PROGRAM  EXECUTIVE.  OFFICE  JOINT  STRIKE  FIGHTER 
SUBJECT:  Notice  of  Concern-Quality  Assurance  Assessment  of  the  F-35  Program 


(Project  No.  I  )2012-m0T  AD-0003) 


We  are  issuing  this  Notice  of  Concern  to  inform  you  that  the  Department  of  Defense, 
Inspector  General  (Dol)  IG)  team  identified  an  issue  that  requires  attention.  During  u  pre- 
assessment  visit  at  Honeywell,  Phoenix,  A/.,  on  July  1 9, 2012,  we  found  that  DCMA  Phoenix 
has  not  been  delegated  oversight  of  production  from  DCMA,  Lockheed  Martin  Fort  Worth.  This 
lack  of  delegation  could  result  in  quality  escapes  as  DCMA  surveillance  and  quality  oversight 
functions  are  not  being  performed.  It  should  be  noted  that  DCMA  Phoenix  docs  have  a 
delegation  tor  repair  and  rework, 

Our  finding  is  attached  to  this  memorandum  for  your  review  and  comment. 

DoD  Directive  7650.3  requires  that  recommendations  be  resolved  promptly.  Please  provide 
comments  that  stale  whether  you  agree  or  disagree  with  the  findings  and  recommendation.  If 
you  agree  with  our  recommendation,  describe  what  actions  you  have  taken  or  plan  to  tokc  to 
accomplish  the  recommendation  and  include  the  completion  dales  of  your  actions.  If  you 
disagree  with  the  recommendation  or  any  part  of  it,  please  give  specific  reasons  why  you 
disagree  and  propose  alternative  action  if  appropriate. 

We  should  receive  your  comments  by  August  1 3,  20 1 2.  If  you  consider  any  matters 
exempt  Horn  public  release,  you  should  mark  them  clearly  for  Inspector  General  consideration. 

I  f  possible,  send  a  portable  document  format  (.pdf)  file  containing  your  comments  to 


I.  Copies  of  your  comments  must  huve  the  actual  signature  of  the 


authorizing  official  for  your  organization.  Wc  arc  unable  to  accept  the  /Signed/  symbol  in  place 
of  an  actual  signature.  If  you  arrange  to  send  classified  comments  electronically,  you  must  send 
them  over  the  SRGRRT  Internet  Protocol  Router  Network  (SIPRNRT). 

We  appreciate  the  courtesies  extended  to  the  staff,  Please  direct  questions  to  


Randol|4h  R.  Stone 
Deputy  Inspector  General 
Policy  and  Oversight 


Attachment: 
As  slated 
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JPO  Response  to  NOC  Honeywell  -  Phoenix ,  AZ 


K-35  JOINT  PROGRAM  OFFICE 
200  121"  Slntl  South,  Suite  600 
Arlington,  Virginia  22202-5402 


From:  Program  Executive  Officer,  F-35  Lightning  II 

To:  Deputy  Inspector  General,  Policy  and  Oversight 

Department  of  Defense  Inspector  General 

Subj :  RESPONSE,  NOTICE  OF  CONCERN  -  QUALITY  ASSURANCE 

ASSESSMENT  OF  THE  F-35  PROGRAM 

1.  I  received  and  analyzed  your  26  Jul  2012  Notice  of  Concern- 
Quality  Assurance  Assessment  of  the  F-35  Program  (Project  No. 
D2012-DTOTAD-0003)  regarding  your  visit  to  Honeywell,  Phoenix, 
AZ,  on  19  July  2012.  I  appreciate  your  team's  efforts  to  assess 
conformity  to  F-35  quality  management  systems.  I  view  your 
assessment  as  a  valuable  tool  to  independently  evaluate  the 
areas  for  improvement  within  the  F-35  Program,  with  the  ultimate 
goal  of  producing  a  quality,  timely,  and  cost-effective  weapon 
system  for  the  United  States  and  our  allies. 

2.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the 
Defense  Contract  Management  Agency  (DCMA),  reviewed  the  single 
finding  and  recommendation.  The  JPO  acknowledges  the  finding 
issued  against  DCMA  Ft  Worth,  and  will  collaborate  with  DCMA  to 
ensure  recommendations  are  assessed  and  implemented  to  the 
maximum  practicable  extent.  A  detailed  discussion  of  our  path 
forward  is  contained  at  Appendix  A  and  is  marked  for  Inspector 
General  Consideration. 

3.  We  will  continue  to  work  with  you  and  your  team  during  the 
quality  assurance  assessments  of  the  remaining  selected  Lockheed 
Martin  Aeronautics  subcontractors. 

cLz/aV- 

DAVID  J/  VENLET 

Vice  Admiral,  U.S.  Navy 


Appendix  A 
Copy  to: 

OUSD(ATSL) 

Director,  Engineering 

JSF  Weapon  System  Program  Manager 

Director,  Production 

Director,  Program  Integration 
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F-35  JOINT  PROGRAM  OFFICE 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


FOR  INSPECTOR  GENERAL  CONSIDERATION 


Appendix  A: 

1.  The  F-35  Joint  Program  Office  (JPO) ,  in  concert  with  the 
Defense  Contract  Management  Agency  (DCMA) ,  reviewed  the  single 
finding  and  recommendation.  The  finding  issued  relative  to  DCMA 
Ft  Worth  is  being  actively  addressed.  They  have  added  a 
supporting  Letter  of  Delegation  for  the  Turbo  Machine.  The 
subsequent  Letter  of  Delegation  from  DCMA  Santa  Ana  was  accepted 
by  DCMA  Phoenix  on  09  August  2012.  We  now  have  surveillance  over 
the  entire  PTMS  to  now  include  repair  of  the  Turbo  Machine.  The 
JSF  JPO  will  continue  to  track  the  completion  of  the  DCMA 
Corrective  Action  plan  as  part  of  the  overall  status  of  findings 
from  all  of  the  DoD  IG  Quality  Assurance  Assessments. 
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NOC  BAE  Systems 


INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 
•WOO  MARK  CENTER  DRIVE 

ALEXANDRIA,  VIRGINIA  22360-1500  _ 

OCT  2  2  2012 


MEMORANDUM  FOR  PROGRAM  EXECUTIVE  OFFICE  JOINT  STRIKE  FIGHTER 

SUBJECT:  Notice  of  Concern-Quality  Assurance  Assessment  of  the  F-35  Program 
(Project  No.  D20 1 2-DT0TAD-0003) 

We  are  issuing  this  Notice  of  Concent  to  inform  you  that  the  Department  of  Defense 
Inspector  General  (l)ol)  10)  loam  identified  issues  that  require  your  attention.  During  (lie 
Quality  Assurance  Assessment1  of  the  F-35  program  at  BAE  Systems,  Snmlcsbury,  United 
Kingdom,  from  September  1 8,  2012  to  September  28, 2012,  the  team  identified  82  findings. 

Wc  sorted  the  findings  into  five  broad  categories:  Process  Discipline,  Document  Control, 
Requirements  Flowdown,  Tool  and  Equipment  Control  and  Calibration,  and  Process  Proofing. 
Our  assessment  of  each  of  these  areas  is  as  follows. 

•  Process  Discipline  Processes  are  in  place,  but  focus  should  be  on  attention  to  detail. 

»  Document  Control-Plans,  procedures,  and  records  me  not  always  maintained. 

o  Requirements  Flowdown-Contractual  requirements  are  not  always  correctly  flowed 
down  and/or  implemented. 

•  Tool/Eqnipmcnt  Control/Calibralion- Tools  and/or  equipment  arc  not  properly 
controlled,  oiul  the  calibration  system  had  serious  deficiencies. 

•  Process  Proofing  Processes  are  reliant  on  operator  knowledge  and  training  rather 
than  documented  processes. 

The  DoD  1G  team's  overall  conclusion  is  that  BAH’s  quality  management  system  is  not 
always  implemented  or  sufficiently  defined.  The  production  facility  lacks  tool  controls,  the 
calibration  system  suffers  from  serious  deficiencies  that  warrant  a  complete  review,  and  software 
lesl  efforts  need  increased  rigor  and  management  participation.  I  Inwever,  we  found  dial  the 
design  review  process  is  systematic,  organized,  and  consistently  implemented;  the  life  cycle  test 
effort  was  assessed  to  be  effective;  and  workforce  and  management  dedication,  knowledge, 
skills,  ami  abilities  arc  driving  BAE’s  success. 

Out  findings  me  attached  to  this  memorandum  for  your  review  and  comment. 

DoD  Directive  7650.3  requires  that  recommendations  he  resolved  promptly.  Please  provide 
comments  that  state  whether  you  agree  or  disagree  with  the  findings  and  recommendations,  If 
you  agree  with  our  recommendations,  describe  what  actions  you  have  taken  or  plan  to  take  to 
accomplish  the  recommendations  and  include  the  completion  dates  of  your  actions.  If  you 
disagree  with  the  recommendations  or  any  part  of  them,  please  give  specific  reasons  why  you 

'  Wc  conducted  the  assessment  in  accordance  with  the  Council  of  the  Inspector  General  on 
Integrity  and  Efficiency  Quality  Standards  for  Inspection  and  Evaluation  and  guidance  in 
AS9I01,  Quality  Management  Systems  Audit  Requirements  for  Aviation,  Space,  and  Defense 
Organizations. 
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disagree  and  propose  alternative  action  if  appropriate.  ( )nce  we  complete  our  assessments,  we 
will  provide  a  report  on  our  findings,  along  with  any  additional  matters  that  might  come  to  our 
attention. 


We  should  receive  your  comments  hy  November  2 1 , 20 1 2.  We  normally  include  copies 
ofthe  comments  in  the  report.  It' you  consider  any  matters  lo  be  exempt  flora  public  release,  you 
should  mark  them  clearly  for  Inspector  General  consideration. 


_ If  possible,  send  a  portable  document  format  (.pdf)  file  containing  your  comments  to 

lELgU- Copies  of  your  continents  must  have  the  actual  signature  of  the 
authorizing  official  for  your  organization,  We  are  unable  to  accept  the  /Signed/  symbol  in  place 
of  the  actual  signature.  If  you  arrange  lo  send  classified  comnienls  electronically,  you  must  send 
them  over  the  SECRET  Internet  Protocol  Router  Network  (S1PRNET). 


Randolph  R.  Stone 
Deputy  Inspector  General 
Policy  and  Oversight 


Attachment: 
As  stated 
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JOINT  STRIKE  FIGHTER  PROGRAM 
200  12r"  Street  South,  Suite  600 
Arlington.  Virginia  22202-5402 

From:  Program  Executive  Officer,  F-35  Lightning  II 

To:  Deputy  Inspector  General,  Policy  and  Oversight 

Department  of  Defense  Inspector  General 

Subj :  RESPONSE,  NOTICE  OF  CONCERN  -  QUALITY  ASSURANCE 

ASSESSMENT  OF  THE  F-35  PROGRAM 

1.  I  received  and  analyzed  your  22  Oct  2012  Notice  of  Concern- 
Quality  Assurance  Assessment  of  the  F-35  Program  (Project  No. 
D2012-DT0TAD-0003 )  regarding  your  visit  to  BAE  Systems, 
Samlesbury,  United  Kingdom,  18-28  September  2012,  I  appreciate 
your  team's  efforts  to  assess  conformity  to  F-35  quality 
management  systems,  contractual  quality  clauses,  and  internal 
processes  and  procedures .  I  view  your  assessment  as  a  valuable 
tool  to  independently  evaluate  the  areas  for  improvement  within 
the  F-35  Program,  with  the  ultimate  goal  of  producing  a  quality, 
timely,  and  cost-effective  weapon  system  for  the  United  States 
and  our  allies. 

2.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the 
Defense  Contract  Management  Agency  (DCMA) ,  the  United  Kingdom 
Ministry  of  Defence  (MOD) ,  BAE,  and  the  prime  contractor, 
Lockheed  Martin  Aeronautics  (LMA) ,  reviewed  each  of  the 
assessment's  findings  and  recommendations.  I  accept  all  of  the 
findings  issued  relative  to  BAE  and  LMA  as  valid.  I  agree  that 
all  of  the  recommendations  issued  relative  to  BAE  and  LMA  should 
be  assessed  and  implemented  to  the  maximum  practicable  extent. 
The  JPO  acknowledges  the  finding  issued  against  DCMA  Northern 
Europe,  and  will  collaborate  with  DCMA  to  ensure  recommendations 
are  assessed  and  implemented  to  the  maximum  practicable  extent. 

I  accept  the  finding  and  recommendation  relative  to  the  Joint 
Program  Office  as  valid.  A  detailed  discussion  of  our  path 
forward  is  contained  at  Appendix  A  and  is  marked  for  Inspector 
General  Consideration. 

3.  We  will  continue  to  work  with  you  and  your  team  during  the 
quality  assurance  assessments  of  the  remaining  selected  F-35 
suppliers . 


Vice  Admiral,  U.S.  Navy 
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Appendix  A 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


Copy  to: 

OUSD ( AT&L) 

Director,  Engineering 

JSF  Weapon  System  Program  Manager 

Director,  Production 

Director,  Program  Integration 
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JOINT  STRIKE  FIGHTER  PROGRAM 
200  12nl  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


FOR  INSPECTOR  GENERAL,  CONSIDERATION 
Appendix  A: 

1.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the 
Defense  Contract  Management  Agency  (DCMA) ,  the  United  Kingdom 
Ministry  of  Defence  (MOD),  BAE,  and  the  prime  contractor, 
Lockheed  Martin  Aeronautics  (LMA) ,  reviewed  each  of  the 
assessment's  findings  and  recommendations.  I  accept  all  of  the 
findings  issued  relative  to  BAE  and  LMA  as  valid.  I  agree  that 
all  of  the  recommendations  issued  relative  to  BAE  and  LMA  should 
be  assessed  and  implemented  to  the  maximum  practicable  extent. 

I  believe  that  every  finding  has  a  deficiency  that  needs  to  be 
further  addressed  by  our  program  office.  However,  we  may 
discover  during  our  root  cause  analysis  and  development  of 
corrective  action  plans  that  a  particular  aspect  of  a  finding 
may  already  be  satisfactorily  addressed  and  fixed.  Further,  I 
intend  to  ensure  that  any  corrections  and  improvements  do  not 
stay  restricted  to  one  location  but  are  distributed  and 
implemented  throughout  the  JSF  enterprise . 

2.  Regarding  the  findings  issued  relative  to  BAE,  the  F-35  JPO 
has  established  a  recurring  joint  working  group  among  the  F-35 
JPO,  BAE,  LMA,  MOD  and  DCMA  to  ensure  that  all  of  the  findings 
and  recommendations  are  understood  and  to  develop  corrective 
action  plans.  Points  of  Contact  within  each  of  the  four 
organizations  were  established  for  every  finding  on  5  November 
2012,  and  discussions  have  commenced.  The  F-35  JPO  has  worked 
with  DCMA  at  BAE  to  understand  and  properly  characterize  each 
finding  and  DCMA  has  issued  Corrective  Action  Requests  (CARs) 
for  appropriate  findings.  The  DCMA  CAR  process  provides  the 
Government  a  systematic  approach  to  ensure  that  corrective 
actions  are  taken  in  a  timely  manner.  The  F-35  JPO,  in  concert 
with  the  responsible  DCMA  organization,  MOD,  and  LMA,  will 
implement  monthly  adjudication  meetings  with  BAE  to  ensure 
appropriate  progress  is  being  accomplished  in  the  corrective 
action  plan  for  each  finding.  The  first  meeting  with  BAE,  LMA, 
MOD,  and  DCMA  will  be  held  in  December  2012. 

3 .  Findings  issued  relative  to  LMA  have  been  rolled  into  the 
tracking  process  initiated  during  the  initial  visit  to  LMA  and 
will  be  included  in  the  LMA  monthly  tracking  meetings. 
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JOINT  STRIKE  FIGHTER  PROGRAM 
200  12"‘  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


4.  The  finding  relative  to  DCMA  Northern  Europe  (DCMA  NE)  is 
being  actively  addressed.  DCMA  NE  has  been  granted  the  authority 
to  increase  staff  by  2  Engineers  and  2  QA  representatives  in  an 
effort  to  increase  the  number  of  Nonconformance  Material  (NCM) 
actions  reviewed.  As  well,  DCMA  NE  will  submit  an  alternate 
approach  plan,  per  DCMA  policy,  to  DCMA  HQ  Operations 
Directorate  for  approval  for  the  reason  that  100%  review  of  the 
supplier's  NCM  submittals  is  not  feasible  due  to  increases  in 
NCMs  related  to  increases  in  aircraft  production.  This  action 
will  be  completed  by  March  15,  2013.  Regarding  the  issue  of 
failure  to  delegate  Material  Review  Board  (MRB)  authority  to 
DCMA  Representatives  at  BAE  System  (BAES)  suppliers;  DCMA  NE  is 
working  with  DCMA  Ft  Worth  to  make  certain  the  proper  contract 
requirement  is  levied  on  BAES  by  Lockheed  Martin  (LM) .  The 
requirement  levied  on  BAES  will  stipulate  that  all  NCMs  shall  be 
submitted  to  the  onsite  Government  representative  for 
concurrence  and  will  be  obligatory  throughout  the  BAES  supply 
chain.  This  will  allow  DCMA  NE  to  issue  delegations,  reinforced 
by  applicable  contract  documents,  which  will  enable  the  proper 
level  of  oversight  of  Material  Review  Board  processes.  These 
actions  will  be  completed  by  15  March  2013. 


5.  As  the  F-35  Program  defines  its  Critical  Safety  Item  (CSI) 
development  and  implementation  requirements,  the  JPO  will 
continue  to  take  DoDIG's  recommendations  into  consideration. 

The  JPO  plans  to  collaborate  with  DCMA  Headquarters  and  DCMA  Ft 
Worth  to  determine  how  best  to  leverage  the  Engineering  Support 
Activity's  CSI  delegation  (to  LMA  and  DCMA)  on  minor 
nonconformance  and  class  II  engineering  change  proposals  within 
the  CSI  policies  to  ensure  that  adequate  checks  and  balances  are 
in  place  to  support  the  F-35  CSI  program.  The  CSI 
responsibility  delegation  will  be  formally  flowed  down  to  the 
Prime  Contractor  and  DCMA  through  F-35  contracts  and  other 
program  plans  related  to  CSI  and  the  JPO/DCMA  Memorandum  of 
Agreement  (MOA) . 
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INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 
'1000  MARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA  22360  1600 

NUV  2  U  2012 


MEMORANDUM  FOR  PROGRAM  EXECUTIVE  OFFICE  JOINT  STRIKE  FIGHTER 

SUBJECT;  Nolice  of  Concern  Quality  Assurance  Assessment  ortlie  F-35  Program 
(Project  No.  D20 1 2-DTtH  AD-0003) 

Wc  are  issuing  this  Notice  of  Concern  to  inform  you  that  the  Department  of  Defense 
Inspector  General  (DoD  IG)  team  identified  issues  that  require  your  attention.  During  the 
Quality  Assurance  Assessment1  of  the  F-35  program  at  Honeywell  Aerospace,  Yeovil,  United 
Kingdom,  from  October  29, 2012  to  November 2, 2012,  the  team  identified  38  findings. 

We  sorted  tlie  findings  into  live  broad  categories:  Process  Discipline,  Document  Control, 
Requirements  blowdown,  Tool  and  Equipment  Control  and  Calibration,  and  Process  Proofing. 
Our  assessment  of  each  of  these  areas  is  as  follows. 

•  Process  Discipline-Failure  to  follow  established  procedures  was  prevalent  throughout 
most  quality  management  processes  reviewed. 

•  Document  Control  Plans,  procedures,  and  records  are  not  always  maintained. 

•  Requirements  Flowdown-Conlraclual  requirements  are  not  always  correctly  flowed 
down  and/or  implemented. 

•  Tool/Equipmcnt  Conlrol/Culibration  -Tools  and/or  equipment  m  e  not  properly 
controlled. 

•  Process  Proofing-  Production  processes  and  work  instructions  were  not  well-defined 
and/or  incomplete. 

The  DoD  IG  team’s  overall  conclusion  is  that  Honeywell’s  quality  management  system  is 
not  always  implemented  or  sufficiently  defined.  Over  half  (60  percent)  of  the  findings  were  in 
process  discipline,  indicating  a  lack  of  attention  to  detail.  In  addition,  the  design  review  process 
anti  training  system  had  serious  deficiencies  that  warrant  a  complete  review. 

Our  findings  are  attached  to  this  memorandum  for  your  review  ami  comment. 

DoD  Directive  7650.3  requires  that  recommendations  be  resolved  promptly,  Please  provide 
comments  that  state  whether  you  agree  or  disagree  with  the  findings  and  recommendations.  If 
you  agree  with  our  recommendations,  describe  what  actions  you  have  taken  or  plan  to  take  to 
accomplish  the  recommendations  and  include  the  completion  dates  of  your  actions,  If  you 
rlisagree  with  the  recommendations  or  any  part  of  them,  please  give  specific  reasons  why  yon 
disagree  and  propose  alternative  action  if  appropriate.  Once  we  complete  out  assessments,  we 


1  We  conducted  the  assessment  in  accordance  with  the  Council  of  the  Inspector  General  on 
Integrity  and  Efficiency  Quality  Standaids  for  Inspection  and  Evaluation  and  guidance  in 
AS9I0I .  Quality  Management  Systems  Audit  Requirements  for  Aviation,  Spuce,  and  Defense 
Organizations, 
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will  provide  a  report  on  our  findings,  along  with  any  additional  matters  that  might  come  to  our 
attention. 

We  should  receive  your  comments  by  December  21,  2012.  We  normally  include  copies 
of  the  comments  in  the  report.  If  you  consider  any  matters  to  be  exempt  from  public  release,  you 
should  mark  them  clearly  for  Inspector  General  consideration. 

I  f  possible,  send  a  portable  document  format  (.pdf)  file  containing  your  comments  to 

CopiC'  of  y, mii  •  actual  signature  of  the 

authorizing  oflicial  for  your  organization.  We  arc  unable  to  accept  the  /Signed/  symbol  in  place 
of  the  actual  signature.  If  you  arrange  to  send  classified  comments  electronically,  you  must  send 
them  over  the  SECRET  Internet  Protocol  Router  Network  (SIPRNET). 


Deputy  Inspector  General 
Policy  and  Oversight 

Attachment: 

As  stated 
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JPO  Response  to  NOC  Honeywell  -  UK 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 

From:  Program  Executive  Officer,  F-35  Lightning  II 

To:  Deputy  Inspector  General,  Policy  and  Oversight 

Department  of  Defense  Inspector  General 

Subj:  RESPONSE,  NOTICE  OF  CONCERN  -  QUALITY  ASSURANCE  ASSESSMENT  OF 
THE  F-35  PROGRAM 

1.  I  received  and  analyzed  your  20  Nov  2012  Notice  of  Concern-Quality  Assurance  Assessment 
of  the  F-35  Program  (Project  No.  D2012-DT0TAD-0003)  regarding  your  visit  to  Honeywell 
Aerospace,  Yeovil,  United  Kingdom,  29  October  -  2  November  2012.  I  appreciate  your  team’s 
efforts  to  assess  conformity  to  F-35  quality  management  systems,  contractual  quality  clauses, 
and  internal  processes  und  procedures.  1  view  your  assessment  as  a  valuable  tool  to 
independently  evaluate  the  areas  for  improvement  within  the  F-35  Program,  with  the  ultimate 
goal  of  producing  u  quality,  timely,  and  cost-effective  weapon  system  for  the  United  Slates  und 
our  allies. 

2.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the  Defense  Contract  Management 
Agency  (DCMA),  the  United  Kingdom  Ministry  of  Defence  (MOD),  the  F-35  prime  contractor, 
Lockheed  Martin,  and  its  suppliers,  Honeywell,  and  BAE  Systems,  reviewed  each  of  the 
assessment's  findings  and  recommendations.  I  accept  all  of  the  findings  issued  relative  to 
Honeywell  and  BAE  as  valid.  1  agree  that  all  of  the  recommendations  issued  relative  to 
Honeywell  and  BAE  should  be  assessed  and  implemented  to  the  maximum  practicable  extent.  A 
detailed  discussion  of  our  path  forward  is  contained  at  Appendix  A  and  is  marked  for  Inspector 
General  Consideration. 

3.  We  will  continue  to  work  with  you  and  your  team  during  the  quality  assurance  assessments  of 
the  remaining  selected  F-35  suppliers. 


CHRISTOPHER  C.  BOGDAN 
Lieutenant  General,  U.S.  Air  Force 

Appendix  A 

Copy  to: 

OUSD(AT&L) 

Director,  Engineering 

JSF  Weapon  System  Program  Manager 

Director,  Production 

Director,  Program  Integration 
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JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


FOR  INSPECTOR  GENERAL  CONSIDERATION 


Appendix  A; 

I  -  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the  Defense  Contract  Management 
Agency  (DCMA),  the  United  Kingdom  Ministry  of  Defence  (MOD),  the  F-35  prime  contractor, 
Lockheed  Martin  Aerospace  (I.MA),  and  its  suppliers,  Honeywell  and  BAF  Systems,  reviewed 
each  of  the  assessment's  findings  and  recommendations.  I  accept  all  of  the  findings  issued 
relative  to  Honeywell  and  BAE  as  valid.  1  agree  that  all  of  the  recommendations  issued  relative 
to  Honeywell  and  BAE  should  be  assessed  and  implemented  to  the  maximum  practicable  extent 
I  believe  that  every  finding  has  a  deficiency  that  needs  to  be  further  addressed  by  our  program 
office.  However,  we  may  discover  during  our  root  cause  analysis  and  development  of  corrective 
action  plans  that  a  particular  aspect  of  a  finding  may  already  be  satisfactorily  addressed  and 
fixed.  Further,  I  intend  to  ensure  that  any  corrections  and  improvements  do  not  stay  restricted  to 
one  location  but  are  distributed  and  implemented  throughout  the  JSF  enterprise. 

2.  Regarding  the  findings  issued  relative  to  Honeywell,  the  F-35  JPO  has  established  a  recurring 
joint  working  group  among  the  F-35  JPO,  Honeywell,  BAE,  LMA,  MOD  and  DCMA  to  ensure 
all  of  the  findings  and  recommendations  arc  understood  and  to  start  developing  corrective  action 
plans.  Points  of  Contact  within  each  of  the  four  organizations  have  been  established  for  every 
finding  and  discussions  have  already  commenced.  The  F-35  JPO  has  begun  working  with 
DCMA  at  Honeywell  to  understand  and  properly  characterize  each  finding  with  an  eye  towards 
drafting  Corrective  Action  Requests  (CARs)  for  appropriate  findings  (on-going,  ECD  December 
2012).  The  DCMA  CAR  process  provides  the  Government  a  systematic  approach  to  ensure  that 
corrective  actions  arc  taken  hi  a  timely  manner.  The  F-35  JPO,  in  concert  with  the  responsible 
DCMA  organization,  MOD,  and  Honeywell,  will  implement  monthly  adjudication  meetings  with 
each  location  included  in  the  assessment  to  ensure  appropriate  progress  is  being  accomplished  in 
the  corrective  action  plan  for  each  finding.  The  first  meeting  with  Honeywell,  BAE,  MOD,  and 
DCMA  will  be  held  in  January  2012. 

3.  Regarding  the  finding  issued  relative  to  BAE  Systems;  it  has  been  rolled  into  the  tracking 
process  initiated  during  the  firstvisit  to  BAE  Systems,  Samlcsbury,  United  Kingdom,  and  will  be 
included  in  the  BAE  monthly  tracking  meetings. 
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NOC  UTAS 


INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 
4800  MARK  CENTER  DRIVE 
ALEXANDRIA.  VIRGINIA  22350  1500 


DEC  I  9  2012 


MEMORANDUM  FOR  PROGRAM  EXECUTIVE  OFFICE  JOINT  STRIKE  FIGHTKR 

SUBJECT:  Notice  of  Concern-Quality  Assurance  Assessment  of  the  F-35  Program 
(Project  No.  D20 1 2-DTOTA D-0003 ) 

We  are  issuing  this  Notice  of  Concern  to  inform  you  that  the  Department  of  Defense 
Inspector  General  (DoD  IG)  team  identified  issues  that  require  your  attention.  During  the 
Quality  Assurance  Assessment  of  the  F-35  program  at  United  Technologies  Corporation, 
Aerospace  Systems  (UTAS),  performed  at  the  Independence,  Ohio  and  Fort  Worth,  Texas 
facilities  from  November  27, 2012  to  Decern  her  5,  2012,  the  team  identified  51  findings. 

Wc  sorted  the  findings  into  five  broad  categories:  Process  Discipline.  Document  Control, 
Requirements  Flowdorvn,  Tool  and  Equipment  Control  and  Calibration,  and  Process  Proofing. 
Our  assessment  of  each  of  these  areas  is  as  follows. 

•  Process  Discipline-Failure  to  follow  established  procedures  was  prevalent  throughout 
most  quality  management  processes  reviewed. 

»  Document  Control  Plans,  procedures,  and  records  arc  not  always  maintained. 

•  Requirements  Plowdown-Contractual  requirements  arc  not  always  correctly  flowed 
down  and/or  implemented. 

•  Tool/Equipment  Control/Calibralion  Tools  and/or  equipment  are  not  properly 
controlled. 

•  Process  Proofing-  Production  processes  and  work  instructions  were  not  well-defined 
and/or  incomplete. 

T  he  DoD  IG  team’s  overall  conclusion  is  that  UTAS’s  quality  management  system 
requires  significant  focus  in  a  couple  of  key  areas.  These  areas  include  the  proper 
implementation  of  quality  and  manufacturing  processes  and  the  flow  down  of  F-35  program 
requirements  to  suppliers.  In  addition,  attention  to  documentation  maintenance  needs  to  be  a 
priority. 

Our  findings  arc  attached  to  this  memorandum  for  your  review  and  comment. 

DoD  Directive  7650.3  requires  that  recommendations  be  resolved  promptly.  Please  provide 
comments  that  slate  whether  you  agree  or  disagree  with  the  findings  and  recommendations.  If 
you  agree  with  our  recommendations,  describe  what  actions  you  have  taken  or  plan  to  lake  to 
accomplish  the  recommendations  and  include  the  completion  dales  of  your  actions.  If  you 


1  We  conducted  the  assessment  in  accordance  with  the  Council  of  the  Inspector  General  on 
Integrity  and  Efficiency  Quality  Standards  for  Inspection  and  Evaluation  and  guidance  in 
AS9101,  Quality  Management  Systems  Audit  Requirements  for  Aviation,  Space,  and  Defense 
Organizations. 
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NOC  UTAS  (cont'd) 


disagree  with  the  recommendations  or  any  pail  of  them,  please,  give  specHTc  reasons  why  yon 
disagree  and  propose  alternative  action  if  appropriate. 

If  possible,  send  a  portable  document  format  (.pdf)  file  containing  your  comments  to 


Copies  of  your  comments  must  have  Ihc  actual  signature  of  the 


authorizing  official  for  your  organization.  We  are  unable  to  accept  the  /Signed/  symbol  in  place 
of  the  actual  signature.  If  you  arrange  to  send  classified  comments  electronically,  yon  must  send 
them  over  the  SRCRHT  Internet  Protocol  Router  Network  (SIPRNET). 

We  should  receive  your  comments  by  January  1 6,  201.3.  We  normally  include  copies  of 
the  comments  in  tile  report.  If  you  consider  any  matters  to  be  exempt  from  public  release,  you 
should  mark  them  clearly  tor  Inspector  General  consideration. 

Once  we  complete  our  assessments,  we  will  provide  a  report  on  our  findings,  along  with 
any  additional  matters  that  might  come  to  our  attention.  We  appreciate  the  courtesies  extended 
to  the  staff.  Please  to 


Randolph  R.  Stone 
Deputy  Inspector  General 
Policy  itiid  Oversight 


Attachment: 
As  slated 
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JPO  Response  to  NOC  UTAS 


JOINT  STRIKE  FIGHTER  PROGRAM 
200  12™  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


From:  Program  Executive  Officer,  F-35  Lightning  II 


JAN  1  8  2013 


To:  Deputy  Inspector  General,  Policy  and  Oversight 

Department  of  Defense  Inspector  General 

Subj:  RESPONSE,  NOTICE  OF  CONCERN  QUALITY  ASSURANCE  ASSESSMENT  OF 
THE  F-35  PROGRAM 

1.  I  received  and  analyzed  your  19  Dec  2012  Notice  of  Concern-Quality  Assurance  Assessment 
of  the  F-35  Program  (Project  No.  D2012-DT0TAD-0O03)  regarding  your  visit  to  United 
Technologies  Corporation,  Aerospace  Systems  (UTAS),  performed  at  the  Independence,  Ohio 
and  Fort  Worth,  Texas  facilities,  27  Novcmbcr-5  December  2012.  I  appreciate  your  team’s 
efforts  to  assess  conformity  to  F-3S  quality  management  systems,  contractual  quality  clauses, 
and  internal  processes  and  procedures.  I  view  your  assessment  as  a  valuable  tool  to 
independently  evaluate  the  areas  for  improvement  within  the  F-35  Program,  with  the  ultimate 
goal  of  producing  a  quality,  timely,  and  cost-effective  weapon  system  for  the  United  States  and 
our  allies.  I  take  seriously  my  responsibility  to  ensure  corrections  are  effectively  implemented 
and  the  F-35  program  office,  in  concert  with  OUSD(AT&L),  will  follow  up  at  all  identified 
locations  to  ensure  compliance  with  corrective  action  plans. 

2.  The  F-35  Joint  Program  Office  (JPO),  in  conceit  with  the  Defense  Contract  Management 
Agency  (DCMA),  and  the  F-35  prime  contractor,  I  .ockhccd  Martin  Aerospace  (LMA),  reviewed 
each  of  the  assessment’s  findings  and  recommendations.  1  accept  all  of  the  findings  issued 
relative  to  UTAS  and  LMA.  I  agree  that  all  of  the  recommendations  issued  relative  to  UTAS 
and  LMA  should  be  assessed  and  implemented  to  the  maximum  practicable  extent.  The  JPO 
acknowledges  the  findings  issued  against  DCMA  Cleveland,  and  will  collaborate  with  DCMA  to 
ensure  recommendations  are  assessed  and  implemented  to  the  maximum  practicable  extent.  A 
detailed  discussion  of  our  path  forward  is  contained  at  Appendix  A  and  is  marked  for  Inspector 
General  Consideration. 

3.  We  will  continue  to  work  with  you  and  your  team  during  the  quality  assurance  assessments  of 
the  remaining  selected  F-35  suppliers. 


Lieutenant  General,  U.S.  Air  Force 


Appendix  A 


Copy  to: 
OUSD(AT&L) 
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IOENT  STRIKE  FIGHTER  PROGRAM 
200  12th  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 

FOR  INSPECTOR  GENERAL  CONSIDERATION 

Appendix  A: 

1.  The  F-35  Joint  Program  Office  (JPO),  in  concert  with  the  Defense  Contract  Management 
Agency  (DCMA),  and  the  F-35  prime  contractor,  Lockheed  Martin  Aerospace  (LMA),  reviewed 
each  of  the  assessment’s  findings  and  recommendations.  I  accept  all  of  the  findings  issued 
relative  to  UTAS  and  LMA  as  valid.  I  agree  that  all  of  the  recommendations  issued  relative  to 
UTAS  and  LMA  should  be  assessed  and  implemented  to  the  maximum  practicable  extent.  I 
believe  that  every  finding  has  a  deficiency  that  needs  to  be  further  addressed  by  our  program 
office.  However,  we  may  discover  during  our  root  cause  analysis  and  development  of  corrective 
action  plans  that  a  particular  aspect  of  a  finding  may  already  be  satisfactorily  addressed  and 
fixed.  I  lake  seriously  my  responsibility  to  eusure  corrections  are  effectively  implemented  and 
the  F-35  program  office,  in  concert  with  OUSD(AT&L),  will  follow  up  at  all  identified  locations 
to  ensure  compliance  with  corrective  action  plans.  Further,  I  intend  to  ensure  that  any 
corrections  and  improvements  do  not  stay  restricted  to  one  location  but  are  distributed  and 
implemented  throughout  the  JSF  enterprise. 

2.  Regarding  the  findings  issued  relative  to  UTAS,  the  F-35  JPO  has  established  a  recurring 
joinl  working  group  among  the  F-35  JPO,  UTAS,  I.MA,  and  DCMA  to  ensure  all  of  the  findings 
and  recommendations  arc  understood  and  to  start  developing  corrective  action  plans  Points  of 
Contact  within  each  of  the  four  organizations  have  been  established  for  every  finding  (completed 
15  Jan  1 3)  and  discussions  have  already  commenced.  The  F-35  JPO  has  begun  working  with 
DCMA  at  both  UTAS  locations  and  at  LMA  Fort  Worth  to  understand  and  properly  characterize 
each  finding  with  an  eye  towards  drafting  Corrective  Action  Requests  (CARs)  for  appropriate 
findings  (on-going,  ECD  15  Feb  13).  The  DCMA  CAR  process  provides  the  Government  a 
systematic  approach  to  ensure  that  corrective  actions  are  taken  in  a  timely  manner  The  F-35 
JPO,  in  concert  with  the  responsible  DCMA  organization,  UTAS,  and  LMA,  will  implement 
monthly  adjudication  meetings  with  each  location  included  in  the  assessment  to  ensure 
appropriate  progress  is  heing  accomplished  in  the  corrective  action  plan  for  each  finding.  The 
first  meeting  with  UTAS,  LMA,  and  DCMA  will  be  held  in  March  2013. 

3.  Regarding  the  findings  issued  relative  to  LMA;  they  has  been  rolled  into  the  tracking  process 
initiated  during  the  first  visit  to  LMA,  Fort  Worth,  and  will  be  included  in  the  LMA  monthly 
tracking  meetings. 

4.  The  findings  relative  to  DCMA  Cleveland  are  being  actively  addressed  DCMA  Cleveland  has 
begun  a  scrub  of  actions  identified  in  the  Letters  of  Delegation  from  DCMA  Lockheed  Martin, 
Fort  Worth  to  ensure  that  all  are  being  addressed.  DCMA  Cleveland  is  working  with  the 
contractor  to  improve  the  flow  of  waivers/deviations,  ensuring  adequate  reviews  and 
recommendations  are  provided  to  the  delegating  authority.  Regarding  the  finding  on 
redelegation  of  Material  Review  responsibilities,  DCMA  Cleveland  is  assessing  all 
documentation  to  ensure  that  correct  delegations  are  in  place.  These  actions  will  be  completed 
by  15  April  2013 
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Appendix  D 

F-35  Joint  Program  Office  and  Defense  Contract 
Management  Agency  Comments 


F-35  UGHTIN1NG  II  JOINT  PROGRAM  OFFICE 
200  12'"  Street  South,  Suite  600 
Arlington,  Virginia  22202-5402 


Mr.  Randolph  R.  Stone  AUG  23  2013 

Deputy  Inspector  General  Policy  and  Oversight 
4800  Mark  Center  Drive 
Alexandria,  Virginia  22350-1500 

Dear  Mr.  Stone, 

Thank  you  for  tire  opportunity  to  comment  on  your  draft  report,  "Quality  Assurance 
Assessment  of  (he  F-35  Lighting  II  Program”,  dated  5  August  2013. 

Roth  the  F-35  Joint  Program  Office  (JPO)  and  the  Defense  Contract  Management 
Agency  (DCMA)  lake  very  seriously  the  findings  and  recommendations  you  offer  in  your  report. 
We  appreciate  the  DoD  IG’s  efforts  to  improve  the  F-35  Program  and  found  the  DoD  IG  Team 
to  be  both  professional  and  knowledgeable.  As  is  noted  in  Enclosure  I ,  F-35  JPO  and  DCMA 
Response  to  Findings/Recommendations,  we  have  been  aggressively  addressing  these  findings 
since  the  beginning  of  the  assessment.  The  JPO  and  DCMA  have  initiated  343  Corrective 
Action  Plans  (CAPs)  to  address  all  363  findings.  As  of20  August  2013,  260  CAPs  for  specific 
contractor  findings  have  been  validated  and  closed,  with  73  CAPs  in  work,  and  10  CAPs  to  be 
approved.  We  are  implementing  corrective  actions  with  the  ultimate  goal  of  producing  a  quality, 
timely,  and  cost-effective  weapon  system  for  the  United  States  and  our  allies. 

Enclosure  1  provides  a  detailed  response  for  each  finding  and  recommendation. 
Enclosure  2  provides  recommended  changes  to  verbiage  in  the  draff  report. 

The  action  officers  for  this  response  are  Mr.  Tim  Traycrs,  Um.travenjfiaisf.mil  (703)  601- 
5650  and  Col  Alex  Stathopoulos,  alcx.statliopoulog@dcma.mil.  (817)  763-4422. 

Sincerely, 


CHRISTOPHER  C.  BOGDAN 
Lieutenant  General,  USAF 
Program  Executive  Officer 


Chief  Operations  Officer 

Defense  Contract  Management  Agency 


Enclosures: 

1 .  F-35  JPO  and  DCM  A  Response  to  Findings/Recommendalions 
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F-35  Joint  Program  Office  and  Defense  Contract  Management 
Agency  Comments  (cont'd) 


2.  F-35  IPO  commcnts/rccommendcd  changes 

cc: 

OUSD  (AT&1-) 

Assistant  Secretary  of  the  Navy  (Research,  Development  &  Acquisition) 
SAF  AQ  Principal  Deputy 
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F-35  Joint  Program  Office  and  Defense  Contract  Management 
Agency  Comments  (cont'd) 


DEFENSE  CONTRACT  MANAGEMENT  AGENCY 

3901  A  AVENUE,  BUILDING  10500 
Fort  LEE,  VIRGINIA  23801 


August  29,  2013 


MEMORANDUM  FOR  DEPARTMENT  OF  DEFENSE,  INSPECTOR  GENERAL,  AUDIT 
POLICY  AND  OVERSIGHT 

SUBJECT:  Draft  Report  Quality  Assurance  Assessment  of  the  F-35  Lightning  II  Program 
(Project  No.  D2012-DTOTAD-0003.000)  dated  August  5,  2013 

Reference:  DoDIG  Draft  Report  Project  No.  D2012-DTOTAD-0003 

We  have  attached  the  Headquarters,  Defense  Contract  Management  Agency’s  comments 
and  documentation  as  requested. 

The  Point  of  contact  for  this  our  response  is  Col  Stathopoulos  at  (817)  763-4422  or 
Alex.Stathopoulos@dcma.mil. 


Attachment: 
As  stated 
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F-35  Joint  Program  Office  and  Defense  Contract  Management 
Agency  Comments  (cont'd) 


Quality  Assurance  Assessment  of  the  F-35  Lightning  II  Program 
Project  No.  D2012-DT0TAD-0003.000~ 

F-35  JPO  and  DCMA  Findings,  Recommendations  and  Responses 

Finding  A:  The  F-35  Joint  Program  Office  (JPO)  did  not  ensure  that  Lockheed  Martin  and  its 
subcontractors  were  applying  rigor  to  design,  manufacturing,  and  quality  assurance  processes. 

F-35  JPO  Response:  Agree 

A  rigorous  design,  manufacturing,  and  quality  assurance  process  exists.  The  Program  has 
continued  to  follow  the  DoD  Systems  Engineering  process  that  spans  from  development  through 
sustainment.  Program  plans  such  as  the  F-35  Systems  Engineering  Program  Plan,  F-35  Program 
Manufacturing  Plan,  and  F-35  Quality  Management  Plan  were  written  by  the  contractor, 
reviewed  by  JPO,  and  have  been  in  place  since  early  in  the  Systems  Development  and 
Demonstration  (SDD)  phase  and  continue  to  evolve  as  the  Program  matures.  Program  reviews 
such  as  Critical  Design  Reviews,  Production  Readiness  Reviews,  and  various  Quality  Assurance 
reviews  (for  example  First  Article  Inspections)  have  been  put  in  place  with  the  F-35  Joint 
Program  Office  and  DCMA  participation.  Major  reviews  have  detailed  entrance  and  exit  criteria 
to  ensure  requirements  are  defined,  implemented,  traceable,  and  verified  by  Program  and 
Department  of  Defense  Subject  Matter  Experts  (SMEs)  participating  in  the  major  reviews 
(providing  both  internal  and  independent  assessments). 

The  F-35  baseline  design  has  continued  to  evolve  as  a  result  of  testing,  producibility 
improvements,  and  affordability.  Contractor  and  Government  change  boards  are  in  use.  Design 
change  management  has  also  evolved  with  the  newly  stood  up  Concurrency  Integrated  Product 
Team  (1PT)  which  ensures  that  proper  systems  engineering,  production,  and  sustainment  reviews 
have  been  conducted  and  the  change  can  be  incorporated  to  meet  cost,  schedule,  and 
performance  requirements. 

The  Manufacturing  plan  has  continued  to  evolve  largely  due  to  on-going  concurrent  design 
development  and  transition  to  rate  production  lines.  Policies  and  procedures  are  in  place  to 
provide  checks  and  balances  in  upgrading  the  manufacturing  operations,  production  floor  layouts 
and  tooling,  and  procurement  operations  to  ensure  maintaining  product  integrity.  Findings  of 
independent  audits  by  contractors,  DCMA,  and  DoD  IG  highlight  opportunities  for 
improvement.  These  audits  and  DCMA  inspections  have  helped  ensure  greater  discipline  in 
adhering  to  their  policies  and  procedures  already  established.  Additionally,  the  Defense  Contract 
Management  Agency  (DCMA)  had  issued  a  Level  III  Corrective  Action  Request  (CAR)  on  July 
2010  for  action  to  be  taken  in  ensuring  process  discipline  and  procedural  adherence. 


Recommendation  A1 :  (JPO)  Ensure  compliance  to  AS9100  throughout  the  F-35  supply  chain. 
F-35  JPO  Response:  Concur 

AS9100  (Quality  Management  Systems  -  Requirements  for  Aviation,  Space,  and  Defense 
Organizations)  is  an  F-35  contractual  requirement.  The  prime  contractor  (Lockheed  Martin 
(LM))  is  AS9100  certified  and  has  flowed  down  this  quality  requirement  to  its  supply  chain. 
AS9100  certification  is  granted  by  a  third  party  accredited  entity.  Periodic  reviews  are  held  by 
that  third  party  entity  to  maintain  certification.  Annual  self  audits  are  conducted  by  both  the 
prime  contractor  and  its  suppliers  to  monitor  compliance  within  their  respective  facilities.  The 

DISTRIBUTION  STATEMENT  A.  Approved  for  public  release;  distribution  is  unlimited. 
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F-35  Joint  Program  Office  and  Defense  Contract  Management 
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Quality  Assurance  Assessment  of  the  F-35  Lightning  II  Program 
Project  No.  D2012-DTOTAD-0003.00<T 
F-35  JPO  and  DCMA  Findings,  Recommendations  and  Responses 

Government  has  had  limited  insight  to  results  of  these  self  audits,  however,  DCMA  performs 
onsite  process  reviews  as  part  of  its  oversight  activity.  DCMA  at  Lockheed  Martin  Fort  Worth 
reorganized  in  February  2013  to  provide  more  focus  and  inspection  emphasis  to  ensure  AS9100 
compliance  and  a  quality  product.  Additionally,  JPO  reviews  AS9100  certification  status  of 
critical  suppliers  during  their  annual  Production  Readiness  Reviews  (PRR).  These  reviews  serve 
as  a  LRIP  risk  management  tool  conducted  with  the  prime  contractor  and  selected  major  critical 
suppliers,  covering  Technical,  Production  and  Sustainment  requirements.  Suppliers  must 
demonstrate  consistent  adherence  of  product  compliant  to  contractual  (AS9100)  and  applicable 
statutory  and  regulatory  requirements.  Findings  of  this  IG  audit  have  caused  both  contractor  and 
Government  entities  to  adjust  the  frequency  and  scope  of  independent  audits  to  ensure  greater 
insight  to  AS9100  compliance. 

Recommendation  A.2:  Ensure  that  Lockheed  Martin  approves  all  design  and  material  review 
board  changes  and  variances  with  Government  concurrence. 

K-35  JPO  Response:  Concur 

Current  F-35  Program  plans  (such  as  the  Configuration  Management  Plan)  require  that  all  major 
design  changes  and  variances  for  production  aircraft  receive  Government  concurrence.  The 
Government  maintains  limited  insight  of  the  contractor’s  Material  Review  Board  (MRB) 
process.  Subsequently,  the  Government  has  reprioritized  and  refined  its  engagement  in  that 
process  to  ensure  improved  effectiveness.  For  nonconformances  that  require  MRB  disposition, 
JPO  delegates  the  approval  of  Class  II  (minor)  changes  to  DCMA  and  reviews  all  Class  I 
changes  that  affect  form,  fit,  or  function.  A  Class  I  change  request  is  dispositioned  through  the 
JPO  Change  Review  Board  (JCRB),  which  performs  a  technical  and  business  assessment  and 
provides  a  written  recommendation  to  the  Joint  Configuration  Control  Board  (JCCB).  With  the 
input  of  major  stakeholders,  the  JCCB  adjudicates  the  recommended  change  and  provides  that 
decision  to  Contracts,  who  notifies  the  Prime  Contractor  to  proceed  in  incorporating  the  change. 

Any  subcontractor  submittal  for  variances  and  engineering  change  requests,  for  which  Lockheed 
Martin  is  the  sole  design  authority,  must  be  go  through  Lockheed  Martin’s  change  control  boards 
for  review  and  approval.  Furthermore,  Lockheed  Martin  must  provide  concurrence  for 
engineering  changes  relating  to  subcontractor  owned  designs.  Various  configuration 
management  checks  and  balances  (such  as  Physical  and  Functional  Configuration  Audits)  are  in 
place  to  ensure  control. 

Recommendation  A.3:  (JPO)  Perform  process  proofing  of  all  critical  processes  to  include  first 
article  inspections. 

F-35  JPO  Response:  Partially  Concur 

The  Government  does  not  have  the  resources  to  perform  process  proofing  of  all  critical 
processes.  Responsibility  and  accountability  rests  on  the  prime  contractor,  with  oversight  from 
the  Government.  Process  proofing  and  First  Article  Inspections  are  Program  Plan  and 
contractual  requirements.  The  Government  performs  audits  of  process  compliance  to  ensure  the 

DISTRIBUTION  STATEMENT  A.  Approved  for  public  release;  distribution  is  unlimited. 
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contractor  is  controlling  critical  operations.  The  Government  closely  monitors  scrap,  rework, 
and  repair  to  ensure  the  contractor  is  conducting  proper  root  cause  analyses  to  implement 
appropriate  corrective  actions.  The  Government  also  monitors  the  quality  of  delivered  product  to 
ensure  control  of  escaped  defects. 

JPO  requires  the  contractor  to  perform  First  Article  Inspections  (FAI)  and  process  proofing  as 
part  of  their  implementation  of  AS9100  and  the  F-35  Program  Quality  Management  Plan.  Over 
27,500  original  baseline  FAIs  have  been  completed  and  approximately  400  more  delta  FAIs  will 
be  performed  for  parts  due  to  concurrency,  producibility,  process  changes,  etc.  In  addition,  the 
prime  contractor  has  implemented  variation  management,  which  is  an  advanced  quality  system 
technique  that  focuses  on  defect  prevention  and  continuous  improvement  through  the 
identification  of  key  product  and  process  characteristics.  This  allows  greater  control  of  the 
manufacturing  process. 

Recommendation  A.4:  (JPO)  Modify  its  contracts  to  include  a  quality  escape  clause,  to  ensure 
the  Government  does  not  pay  for  nonconforming  product. 

F-35  JPO  Response:  Concur 

JPO  concurs  that  greater  incentives  should  be  taken  to  preclude  the  cost  of  poor  quality.  Major 
non-conformances  that  do  not  meet  engineering  specification  are  identified  and  adjudicated  for 
acceptance  or  rejection  with  considerations/withholds  placed  on  a  variance.  Minor  non¬ 
conformances  are  reworked,  repaired,  or  used  as  is.  The  program  has  transitioned  to  fixed  price 
type  contacts  with  a  0/100  share  ratio  and  clauses  will  prescribe  to  applicable  FAR  requirements. 
Along  with  considerations  /  withholds  on  non-conformances,  this  contract  structure  will  facilitate 
greater  incentive  by  the  contractor  to  provide  quality  assurance  in  order  to  maintain  cost  and 
schedule  obligations  without  over  running  their  negotiated  budget.  The  contractor  will  absorb  a 
level  of  cost  for  poor  quality.  In  addition,  the  JPO  continues  to  establish  quality  performance 
targets  with  commitments  from  JPO  and  LM  senior  leadership.  These  targets  are  based  upon 
continuous  learning  in  fabrication  and  assembly  operations,  corrective  action  implementation,  in 
addition  to  product  and  process  improvements  as  a  result  of  affordability  and  concurrency 
changes. 

Recommendation  A.5:  (JPO)  Assess  the  impacts  and  risks  to  all  delivered  aircraft  for  all 
findings. 

F-35  JPO  Response:  Concur 

The  impacts  and  risks  of  all  findings  were  assessed  as  part  of  the  Corrective  Action  Request 
(CAR)/Corrective  Action  Plan  (CAP)  process.  When  a  CAR  was  written,  the  F-35  fleet  was 
assessed  for  impact.  Concurrence  on  each  CAP  considered  effectivity,  safety,  and  contract 
elements.  This  determined  priority  and  timing  required  to  implement  the  corrective  action  into 
the  affected  aircraft  as  appropriate. 
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Finding  B:  JPO  did  not  flow  down  critical  safety  item  (CSI)  requirements. 

F-35  JPO  Response:  Agree 

Public  Law  108-36,  section  802,  requiring  a  Critical  Safety  Item  program  was  introduced  in 
fiscal  2004  legislation,  three  years  after  the  F-35  SDD  contract  award.  Further,  guidance  from 
DoD  on  how  to  implement  CSI  was  not  immediately  forthcoming.  As  such,  the  F-35  program 
delayed  CSI  implementation  pending  clear  policy  and  to  minimize  unnecessary  cost  growth.  As 
the  name  implies,  the  focus  of  CSI  is  aviation  safety.  The  F-35  program  and  contractor  team 
treat  safety  as  a  Number  1  priority.  The  JPO’s  development  specification  includes  stressing 
criteria  for  aviation  safety  (max  loss  of  aircraft  rate),  hazard  risk  levels,  and  mission  completion 
success  criteria.  Further,  LM’s  engineering  standard  work  requires  a  comprehensive  product 
integrity  program  to  ensure  the  safety  and  performance  of  safety  /  mission  critical  components, 
subsystems,  and  assemblies.  Rigorous  guidelines  to  include  redundancy,  backup  systems,  design 
and  qualification  standards,  etc.  are  imposed  on  these  components  to  ensure  safety  and 
performance.  Many  of  the  prescribed  CSI  practice,  such  as  100%  inspection,  are  subsumed 
under  the  umbrella  of  the  integrity  programs:  fracture  critical;  safety  critical;  durability  critical; 
and  mission  critical.  Nonetheless,  JPO  and  the  contractor  developed  and  regularly  update  a  list 
of  CSI  parts  and  assemblies.  This  list  has  been  required  on  each  LRIP  contract  to  date.  That 
said,  the  JPO  is  phasing  in  the  remaining  tenants  of  a  CSI  program  through  ongoing  LRIP 
contracting  actions  and  new  SDD  actions.  For  example,  recent  closure  on  the  LRIP  6  contract 
includes  additional  CSI  related  tasks  for  variance  management  and  notification. 


Recommendation  B.l:  (JPO)  Implement  an  aviation  critical  safety  item  program  that  meets  the 
requirements  of  the  Public  Law  and  DoD  Policy,  which  would  include  flow  down  of 
requirements  for  a  critical  safety  item  program  to  Lockheed  Martin  and  its  subcontractors. 

F-35  JPO  Response:  Concur 

JPO  is  working  with  the  prime  contractor  in  developing  a  Request  for  Proposal  (RFP)  for  a  two- 
phased  CSI  Non  Recurring  Engineering  (NRE)  approach.  This  will  require  the  development  of 
comprehensive  CSI  requirements  which  will  be  added  to  the  SDD  contract’s  Statement  of  Work. 
These  requirements  will  address  the  identification  of  critical  characteristics  that  enable 
compliance  at  the  prime  contractor  and  throughout  its  supply  base.  Once  the  SDD  NRE  efforts 
are  completed,  the  full  scope  of  the  F-35  CSI  program  will  be  included  in  future  production  and 
sustainment  contracts.  Commencement  of  certain  initial  recurring  tasks  will  be  implemented 
using  the  LRIP  6  contract,  hi  the  interim,  the  contractors’  integrity  and  quality  programs  are 
providing  assurance  for  continued  aviation  safety. 

Recommendation  B.2:  (JPO)  Assess  the  impacts  and  risks  to  all  delivered  aircraft  for  critical 
safety  item  deficiencies. 
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F-35  JPO  Response:  Concur 

Deficiencies  and  impacts  to  critical  parts  are  identified  and  assessed  during  quality  assurance 
practices  and  testing  operations  as  part  of  the  SDD  design  and  product  verification  processes.  If 
safety  issues  and  limitations  are  discovered,  changes  are  initiated  and  incorporated  as  warranted 
to  the  test  and  production  aircraft.  Initial  and  continued  airworthiness  is  a  major  focus  of  the  F- 
35  Class  Desk,  Airworthiness  Team,  Integrated  Test  Team,  and  Integrated  Product  Teams.  The 
SDD  test  aircraft  have  more  than  5,000  flight  hours.  The  combined  F-35  fleet  has  accumulated 
more  than  7,000  flight  hours.  This  is  not  to  say  that  there  are  no  significant  issues  or  discoveries. 
Rather,  there  is  an  upfront  understanding  whether  any  new  discovery  poses  an  increased  safety 
risk.  If  necessary,  proper  actions  are  taken,  such  as  suspending  flight  operations,  directing 
inspections,  adding  flight  limitations  or  restrictions,  and  directing  hardware  or  software 
modifications  to  mitigate  the  safety  risks. 


Finding  C:  JPO  did  not  ensure  that  Lockheed  Martin  flowed  down  quality  assurance  and 
technical  requirements  to  subcontractors. 

F-35  JPO  Response:  Partially  Agree 

Processes  are  in  place,  which  ensure  that  requirements  are  flowed  down  from  the  prime 
contractor  to  its  supply  base.  But,  there  are  areas  for  improvement.  A  significant  number  of  IG 
findings  were  noted  in  this  area.  The  four  key  areas  of  highest  concern  were  First  Article 
Inspection,  Configuration  Control,  Design  and  Development  Requirements,  and  Diminishing 
Manufacturing  Supply  and  Materiel  Shortages. 

First  Article  Inspection  (FAI)  -  Although  the  IG  finding  states  that  there  is  a  lack  of  definitive 
FAI  requirement  flowed  down  to  the  prime  contractor’s  supply  chain,  all  major  suppliers  are 
required  to  adhere  to  AS9100  in  which  FAI  is  addressed.  AS9102,  First  Article  Inspection,  is 
used  as  a  guideline  to  assist  in  meeting  the  AS9100  requirement.  Validation  of  interface  control 
features,  key  characteristics,  interchangeability  /  replaceability,  outer  mold-line,  coating 
application,  and  component  end  item  inspections  are  conducted  throughout  major  assemblies  and 
tied  to  the  aircraft  acceptance  (DD250)  process.  As  such,  LM  does  not  require  FAls  to  be 
performed  on  major  component  assemblies  such  as  the  wing  and  the  center,  forward,  and  aft 
fuselages. 

Configuration  Control  -  The  associated  IG  finding  states  that  there  is  a  lack  of  requirement  flow 
down  and  communication  between  Lockheed  Martin  and  its  suppliers,  leading  to  configuration 
management  issues.  The  F-35  maintains  a  detailed  Configuration  Management  Plan  which 
describes  configuration  control  through  the  prime  contractor  and  its  supply  chain.  Check  and 
balances  have  been  put  in  place  to  ensure  configuration  control.  These  checks  and  balances 
include  (but  are  not  limited  to)  Functional/Allocated/Product  Baseline  Reviews,  First  Article 
Inspections,  Physical  Configuration  Reviews  /  Audits,  Function  Configuration  Audits,  and 
System  Verification  Reviews.  Preparation  for  these  major  reviews  serves  as  a  means  to 
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communicate  and  understand  the  requirements  between  the  prime  contractor  and  its  suppliers. 
Execution  of  such  reviews  validates  compliance  and  ensures  configuration  control. 

Design  and  Development  Requirements  -  A  key  finding  in  this  area  dealt  with  a  lack  of  flow 
down  of  Mission  Systems  Software  Development  requirements,  in  addition  to  verification  and 
traceability  of  those  requirements.  It  is  true  that  there  are  challenges  with  traceability  of  mission 
systems  requirements.  However,  it  is  not  accurate  to  say  that  the  requirements  are  untraceable. 
Traceability  can  be  accomplished.  The  challenge  is  that  the  traceability  requires  access  to 
several  tools  and  is  more  manual  in  nature  and  not  an  automatic  Dynamic  Object  Oriented 
Requirements  System  (DOORs)  function.  The  finding  also  mentioned  the  lack  of  maintenance 
of  the  Air  System  Block  Plan.  This  statement  is  inaccurate.  At  some  point  in  time  this  statement 
may  have  been  true,  however,  the  Air  System  Block  Plan  is  maintained  via  a  Contract  Data 
Requirements  List  (CDRL)  that  is  updated  every  6  months  and  the  Program  now  has  a  Build 
Review  Board  that  actively  manages  the  capability  content  of  the  detailed  build  plans  that 
constitute  the  higher  level  Block  Plans.  The  Build  Review  Board  monitors  execution  progress 
and  planning  of  new  software  builds  to  ensure  that  capabilities  are  delivered  on  time  based  on  a 
realistic  schedule  and  acts  as  the  Decision  Body  that  makes  informed  trades  to  support  software 
execution  and  LRIP  production  deliveries. 

Diminishing  Manufacturing  Supply  and  Materiel  Shortages  (DMSMS)-  This  finding  stated  that 
JPO  had  not  funded  Lockheed  Martin  (LM)  to  implement  a  DMSMS  program.  The  conversion 
from  funding  DMSMS  on  the  SDD  contract  to  the  LRIP  contracts  was  taking  place  when  the  1G 
was  briefed.  During  this  interim,  DMSMS  was  not  funded  by  the  F-35  Joint  Program  Office. 
Potential  risks  were  mitigated  as  a  result  of  the  prime  contractor  executing  on  internal  company 
funding  and  converting  to  an  Undefinitized  Contract  Authorization  (UCA),  with  Full 
Authorization  funded  under  LRIP  6.  In  addition,  the  DMSMS  strategy  was  under  review,  the 
charter  was  signed,  and  the  management  plan  was  in  draft  by  the  F-35  Joint  Program  Office. 

The  prime  contractor  also  revised  the  Supply  Chain  Management  Plan  which  now  includes  the 
DMSMS  Team  as  a  stakeholder,  ensuring  proper  coordination.  Furthermore  the  DMSMS 
Supplier  Statement  of  Work  requirement  was  updated  to  specifically  require  LM  suppliers  to 
flow  down  DMSMS  requirements  to  sub-tier  suppliers.  Funding  and  improvements  concerning 
DMSMS  requirements  have  allowed  better  proactive  parts  obsolescence  management. 

Recommendation  C:  (JPO)  Perform  technical  and  quality  assurance  requirement  flow  down 
and  verification  throughout  the  F-35  supply  chain. 

F-35  JPO  Response:  Partially  Concur 

The  Government  does  not  have  the  responsibility  or  resources  to  perform  requirement  flow  down 
verification  throughout  the  prime  contractor’s  entire  supply  chain.  The  prime  contractor  is 
responsible  and  accountable,  with  oversight  from  the  Government.  The  Government  performs 
spot  checks  and  conducts  surveillance  to  ensure  the  contractor  is  flowing  down  requirements  to 
its  subcontractors.  The  Government  monitors  this  closely  through  participation  in  Design 
Reviews,  Production  Readiness  Reviews,  Structural  Equivalency  Reviews,  Program 
Management  Reviews,  production  floor  inspections,  configuration  audits,  etc.  On  site 
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surveillance  is  primarily  performed  by  DCMA  to  ensure  the  contractor  and  its  subcontractors  are 
complying  with  contractual  requirements. 


Finding  D:  JPO  did  not  establish  an  effective  quality  assurance  organization. 

F-35  JPO  Response:  Agree 

The  F-35  Program  management  team  is  committed  to  establishing  an  effective  Quality 
Assurance  program  and  organization.  It  has  empowered  its  Quality  Team,  under  the  Production 
IPT,  to  define  measurable  Program  quality  requirements  and  is  engaged  in  creating  a  program 
environment  that  fosters  continuous  improvement.  JPO’s  Quality  Team  is  engaged  during 
contract  and  product  development  to  ensure  quality  considerations  are  considered.  Their 
engagement  continues  through  fabrication  and  assembly  to  ensure  quality  control  and  continuous 
improvement  through  aircraft  delivery.  JPO  has  integrated  DCMA  as  part  of  its  Quality  Team, 
per  the  long  standing  and  ongoing  Program  IPT  structure  and  the  Program’s  Memorandum  of 
Agreement  (MOA)  with  DCMA,  to  enhance  quality  surveillance  and  performance  monitoring. 
DCMA  serves  as  an  on-site  agent  and  is  responsible  for  contract  administration  and  has 
resources,  training,  and  processes  to  effectively  administer  quality  assurance.  Their  insight  and 
oversight  of  contractor  practices  places  them  in  the  best  position  to  identify,  report,  and  address 
quality  assurance  issues.  Based  upon  some  of  the  IG  findings,  JPO  and  DCMA  are  in  the 
process  of  refining  their  teaming  structure  to  ensure  more  cogent  focus  on  critical  processes  and 
product  quality  results. 

JPO  has  formed  a  partnership  with  Lockheed  Martin  and  DCMA.  Each  entity  maintains  its 
independent  perspective,  but  is  teamed  to  achieve  a  common  goal  -  ensure  the  delivery  of  a 
quality  product  to  US  Services  and  international  Partners’  warfighters.  It  is  through  this 
arrangement  that  requirements  are  articulated,  executed,  verified,  and  controlled.  The 
Government  (JPO  and  DCMA)  continues  to  work  closely  with  the  contractor’s  Program  and 
Core  Quality  organizations  to  ensure  adequate  support  is  provided  to  meet  Program  objectives. 
As  the  Program  transitions  to  Full  Rate  Production,  JPO  acknowledges  the  need  for  a  focused 
quality  organization. 


Recommendation  D:  (JPO)  Establish  an  independent  quality  assurance  organization,  which  has 
the  authority  and  resources  to  enforce  the  AS9100  standard  and  F-35  product  quality. 

F-35  JPO  Response:  Non-concur 

JPO  acknowledges  opportunities  to  improve  management  of  Quality  performance  and  the 
benefits  of  realignment  within  the  F-35  Program  organizational  structure  to  better  support 
transition  from  Low  Rate  Initial  Production  to  Full  Rate  Production.  These  efforts  have 
commenced  to  include  an  increase  in  resources  and  realignment  of  the  Quality  Team  to  report  to 
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program  leadership.  The  JPO  non-concurs  with  establishing  an  independent  quality  assurance 
organization.  DCMA  performs  the  role  of  the  independent  quality  assurance  organization  for  the 
F-35  and  other  DoD  programs.  DCMA  conducts  process  surveillance  against  the  AS9100 
standard  and  enforces  F-35  product  quality  requirements.  Additionally,  the  JPO  Quality  Team  is 
staffed  from  two  independent  organizations;  Air  Force  Life  Cycle  Management  Center 
(AFLCMC)  and  Naval  Air  Systems  Command  (NAVAIR).  While  serving  on  the  JPO  Quality 
Team,  these  individuals  retain  their  responsibility  to  invoke  their  independent  AFLMC  and 
NAVAIR  leadership  in  the  event  the  JPO  does  not  properly  address  quality  concerns. 


Finding  E:  DCMA  was  not  performing  quality  assurance  oversight  commensurate  with  product 
criticality.  Insufficient  written  direction  from  the  JPO  coupled  with  inadequate 
execution  from  DCMA  resulted  in  ineffective  Government  oversight  of  the  F-35 
program. 

F-35  JPO  Response:  Agree 

DCMA  is  an  independent  DoD  component  that  works  directly  with  Defense  suppliers  and 
buying  agencies  to  ensure  contract  compliance.  DCMA  partnership  with  the  JPO  is  crucial  to 
delivery  of  the  F-35.  DCMA’s  oversight  activities  are  focused  on  process  surveillance  to  ensure 
systemic  issues  are  adequately  addressed.  Nonconformances  are  assessed  according  to  severity 
and  recurrence.  Corrective  action  requests  have  been  generated  to  address  nonconformances,  in 
concert  with  communicating  with  JPO  on  the  implementation  of  corrective  actions.  In  addition, 
DCMA  is  an  integral  part  of  the  Material  Review  Board  (MRB)  process.  They  will  continue  to 
review  the  classification  of  all  nonconformance,  raising  issues  which  affect  form,  fit,  or  function 
to  JPO  attention,  and  providing  disposition  to  minor  nonconformance  as  required.  DCMA  also 
ensures  that  the  prime  contractor  and  its  suppliers  are  adequately  employing  the  MRB  process. 
Contractor  performance  status  is  also  provided  monthly  to  JPO  in  functionally-aligned  written 
reports. 

JPO  feedback  is  critical  to  DCMA.  DCMA  also  relies  on  its  Headquarter  reviews  to  measure 
performance  based  upon  agency  policy,  contract  content,  and  customer  expectations.  JPO  and 
DCMA  have  initiated  improvements  in  their  teaming  relationship  to  include  emphasizing 
priorities  and  exploring  the  sharing  of  resources.  DCMA  has  been  conducting  surveillance 
activities  throughout  the  supply  chain  since  the  beginning  of  the  F-35  SDD  contract.  1 37  Letters 
of  Delegation  (LODs)  are  in  place  with  all  major  and  critical  parts  suppliers  and  being 
continually  updated  to  meet  program  needs.  Additionally,  as  the  Critical  Safety  Item  (CSI) 
program  matures,  DCMA  and  JPO  will  continue  refocusing  limited  assets  to  ensure  coverage. 
DCMA  (Ft  Worth)  and  its  delegated  team  will  continue  to  utilize  the  Corrective  Action  Report 
(CAR)  methodology  to  focus  contractor  efforts  and  resolve  risks  while  enhancing 
CAR/Corrective  Action  Plan  communication  with  the  JPO. 
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Recommendation  E.Ia  (JPO):  Revise  the  Defense  Contract  Management  Agency 
memorandum  of  agreement  to  provide  explicit  surveillance  criteria 

F-35  JPO  Response:  Concur 

The  Memorandum  of  Agreement  (MOA),  annex  A-P,  dated  February  2011,  between  the  JPO  and 
DCMA  is  being  updated  to  reflect  Program  objectives  requiring  DCMA  support  in  the  following 
areas:  Management  of  contractor  Quality  Process  implementation,  control,  maturity,  and 
verification;  Material  Review/Change  Board  Process  authority  and  related  task  requirements, 
Foreign  Object  Damage  (FOD)  management,  Government  Flight  Representative  surveillance. 
Safety  of  Flight  inspection  requirements,  contractual  Corrective  Action  Request  notification.  Air 
system  acceptance  (DD250),  Supply  Chain  Management,  etc.  However  specific  surveillance 
activities  and  priorities  are  articulated  via  other  tools,  such  as  Quality  Assurance  Letters  of 
Inspection  (QALI’s). 

It  is  important  to  note  that  the  MOA  update,  though  crucial  in  shaping  organizational  roles  and 
responsibilities  that  address  the  findings  in  this  report,  is  only  part  of  the  corrective  action 
process  being  implemented.  Execution  of  DCMA  policies,  enhanced  communication  between 
the  JPO  and  DCMA,  and  enhanced  inter-DCMA  communications  with  supplier  DCMA 
organizations  all  play  a  role  in  improving  surveillance.  In  addition  to  the  MOA,  DCMA  utilizes 
risk  based  surveillance  plans  to  support  F-35  contracts.  As  part  of  the  functional  surveillance 
planning  process,  DCMA  determines  what  types  of  surveillance  techniques  will  be  applied  to 
ensure  a  supplier  has  adequate  quality  processes  in  place  and  meet  all  contractual  requirements. 
Risks  are  identified  through  historical  knowledge  of  the  suppliers  systems  and  processes,  results 
of  surveillance  execution,  analysis  of  DCMA  internal  data,  supplier  data  and  customer  data. 

Recommendation  E.I.b:  (JPO)  Ensure  that  Defense  Contract  Management  Agency  is 
performing  quality  assurance  oversight  commensurate  with  product  criticality. 

F-35  JPO  Response:  Concur 

DCMA  has  been,  and  continues  to  be,  responsive  to  product  criticality  while  performing  its 
contact  administration  quality  assurance  functions.  This  includes  optimizing  quality  assurance 
oversight  of  contractors’  critical  processes,  operations,  and  product.  Product  criticality  is 
governed  by  Federal  Acquisition  Regulation  derived  surveillance  strategies.  FAR/DFAR 
regulatory  requirements  as  manifest  in  F-35  contracts  are  also  guided  by  DCMA  policies. 

DCMA  policies  define  risk  considerations  and  thresholds  in  developing  surveillance  plans. 
Quality  specialists  determine  what  types  of  surveillance  techniques  will  be  applied  to  ensure  a 
supplier  has  adequate  quality  processes  in  place.  Surveillance  activities  and  data  analysis  allow 
DCMA  to  re-evaluate  risk  levels  and  where  appropriate  adjust  surveillance  plans. 

As  non-conformance  is  identified,  JPO  may  request  special  surveillance  be  performed  in  other 
areas  due  to  severity  or  whether  they  are  systematic  in  nature.  This  is  captured  in  other 
documents  such  as  QALI’s.  The  DCMA  monthly  report  also  is  a  means  of  communicating 
quality  assurance  oversight  as  it  pertains  to  production  status.  Continuous  communication 
between  JPO  and  DCMA  to  discuss  and  resolve  quality  issues  is  critical  to  meeting  Program 
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F-35  Joint  Program  Office  and  Defense  Contract  Management 
Agency  Comments  (cont'd) 


Quality  Assurance  Assessment  of  the  F-35  Lightning  II  Program 
Project  No.  D2012-DT0TAD-0003.000 
F-35  JPO  and  DCMA  Findings,  Recommendations  and  Responses 

objectives.  Performing  quality  assurance  in  accordance  with  product  criticality  is  central  to  how 
DCMA  conducts  contract  surveillance. 

Recommendation  E.2.a:  (DCMA)  Provide  a  comprehensive  quality  assurance  oversight  plan 
for  Joint  Program  Office  approval  to  be  included  in  the  memorandum  of  agreement. 

F-35  DCMA  Response:  Partially  Concur 

DCMA  will  ensure  the  MOA,  as  currently  being  updated,  complements  DCMA  risk  based 
surveillance  planning.  The  DMCA  surveillance  planning  process  is  risk  based  and  developed 
through  historical  knowledge  of  the  suppliers  systems  and  processes,  results  of  surveillance 
execution,  analysis  of  DCMA  internal  data,  supplier  data  and  customer  data. 

DCMA  is  also  implementing  a  dedicated  Supply  Chain  Management  team  at  LMFW  to  address 
the  significant  supply  chain  oversight  challenge  of  the  F-35  program.  The  focus  of  the  team  will 
be  to  evaluate  LMFWs  supply  chain  processes  and  system  as  well  as  a  more  focused  emphasis 
on  risk  suppliers  throughout  the  Supply  Chain. 

Recommendation  E.2b:  (DCMA)  Audit  the  execution  of  the  quality  assurance  oversight  plan 
throughout  the  F-35  supply  chain. 

F-35  DCMA  Response:  Concur 

DCMA  will  comply  with  its  quality  assurance  oversight  plan.  Regular  audits  of  supplier  and 
prime  DCMA  organizations  are  crucial  to  maximizing  the  effectiveness  of  risk  based 
surveillance.  Within  DCMA,  Management  Review  Teams  (MRT)  provide  comprehensive 
functional  reviews  on  our  quality  program.  These  risk-based  audit  efforts,  are  conducted  by 
independent  quality  experts  and  occur  throughout  the  entire  DCMA  organization  ensuring 
coverage  of  the  F-35  supply  chain.  Continued  communication  of  audit  results  with  our 
customers  and  enhanced  communication  between  DCMA  supplier  CMOs  will  also  supplement  a 
robust  audit  process. 
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Acronyms  and  Abbreviations 


ANSI  American  National  Standards  Institute 
AS  Aerospace  Standard 
ATP  Acceptance  Test  Procedure 
BOS  Back-Up  Oxygen  System 
CAR  Corrective  Action  Request 
CDR  Critical  Design  Review 
CSI  Critical  Safety  Item 
CTOL  Conventional  Takeoff  and  Landing 
CV  Carrier-Suitable  Variant 
DCMA  Defense  Contract  Management  Agency 
DMSMS  Diminishing  Manufacturing  Sources  and  Material  Shortages 
DOORS  Dynamic  Object  Oriented  Requirements  System 
DU  Display  Unit 

EMI  Electromagnetic  Interference 
ESD  Electrostatic  Discharge 
EU  Electronics  Unit 
FAI  First  Article  Inspection 

FMECA  Failure  Modes,  Effects,  and  Criticality  Analysis 
FOD  Foreign  Object  Debris 

FRACAS  Failure  Reporting,  Analysis,  and  Corrective  Action  System 
ICD  Interface  Control  Document 
IEC  International  Electrotechnical  Commission 
IPT  Integrated  Product  Team 
ISO  International  Organization  for  Standardization 
JACG  Joint  Aeronautical  Commanders  Group 
JPO  Joint  Program  Office 
JSF  Joint  Strike  Fighter 
LOD  Letter  of  Delegation 
LRIP  Low-Rate  Initial  Production 
LSLM  Limited  Shelf-Life  Materials 
NOC  Notice  of  Concern 
OBOGS  On-Board  Oxygen  Generation  System 
OFI  Opportunity  for  Improvement 
OIG  Office  of  Inspector  General 
PCD  Panoramic  Cockpit  Display 
PDR  Preliminary  Design  Review 
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QN  Quality  Notifications 
SDD  Software  Design  Document 
SDP  Software  Development  Plan 
SOW  Statement  of  Work 
SRS  Software  Requirements  Specification 
STOVL  Short  Takeoff  and  Vertical  Landing 
SVDD  Software  Version  Description 
UTAS  United  Technology  Aerospace  Systems 
VDD  Version  Description  Document 
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Whistleblower  Protection 

U.S.  Department  of  Defense 


The  Whistleblower  Protection  Enhancement  Act  of  2012  requires 
the  Inspector  General  to  designate  a  Whistleblower  Protection 
Ombudsman  to  educate  agency  employees  about  prohibitions  on 
retaliation,  and  rights  and  remedies  against  retaliation  for  protected 
disclosures.  The  designated  ombudsman  is  the  DoD  IG  Director  for 
Whistleblowing  &  Transparency.  For  more  information  on  your  rights 
and  remedies  against  retaliation,  go  to  the  Whistleblower  webpage  at 
www.dodig.mil/programs/whistleblower. 


For  more  information  about  DoD  IG 
reports  or  activities,  please  contact  us: 

Congressional  Liaison 

Congressional@dodig.mil;  703.604.8324 

DoD  Hotline 

800.424.9098 

Media  Contact 

Public.Affairs@dodig.mil;  703.604.8324 

Monthly  Update 

dodigconnect-request@listserve.com 

Reports  Mailing  List 

dodig_report-request@listserve.com 

Twitter 

twitter.com/DoD_IG 


